-- Logs begin at Thu 2022-05-26 17:07:15 UTC, end at Thu 2022-05-26 18:22:49 UTC. -- May 26 17:49:06.794004 dv-cinder-839064-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 26 17:49:06.806002 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 26 17:49:06.808838 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 26 17:49:06 2022] *** May 26 17:49:06.808985 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 26 17:49:06.809089 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 26 17:49:06.809190 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: nodename: dv-cinder-839064-2-iscsi May 26 17:49:06.809283 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: machine: x86_64 May 26 17:49:06.809383 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: clock source: unix May 26 17:49:06.809481 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: pcre jit disabled May 26 17:49:06.809588 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: detected number of CPU cores: 4 May 26 17:49:06.809681 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: current working directory: / May 26 17:49:06.809773 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: detected binary path: /usr/bin/uwsgi-core May 26 17:49:06.809877 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: your processes number limit is 47680 May 26 17:49:06.809978 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: your memory page size is 4096 bytes May 26 17:49:06.810078 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: detected max file descriptor number: 2048 May 26 17:49:06.810222 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: lock engine: pthread robust mutexes May 26 17:49:06.810484 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: thunder lock: enabled May 26 17:49:06.810770 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 26 17:49:06.810951 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 26 17:49:06.859592 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: Python main interpreter initialized at 0x55c8dffda9c0 May 26 17:49:06.859592 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: python threads support enabled May 26 17:49:06.859592 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: your server socket listen backlog is limited to 100 connections May 26 17:49:06.859592 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: your mercy for graceful operations on workers is 90 seconds May 26 17:49:06.859993 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: mapped 403077 bytes (393 KB) for 2 cores May 26 17:49:06.860071 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: *** Operational MODE: preforking *** May 26 17:49:06.860333 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: *** uWSGI is running in multiple interpreter mode *** May 26 17:49:06.860333 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: spawned uWSGI master process (pid: 80248) May 26 17:49:06.860590 dv-cinder-839064-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 26 17:49:06.861730 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: spawned uWSGI worker 1 (pid: 80260, cores: 1) May 26 17:49:06.861730 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: spawned uWSGI worker 2 (pid: 80261, cores: 1) May 26 17:49:06.861730 dv-cinder-839064-2-iscsi devstack@keystone.service[80248]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 26 17:49:12.655352 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 17:49:12.655352 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: from cryptography.utils import int_from_bytes May 26 17:49:12.656153 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 17:49:12.656153 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: from cryptography.utils import int_from_bytes May 26 17:49:12.696324 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 17:49:12.696324 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: from cryptography.utils import int_from_bytes May 26 17:49:12.696923 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 17:49:12.696923 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: from cryptography.utils import int_from_bytes May 26 17:49:12.706916 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 26 17:49:12.707113 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 26 17:49:12.707309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] command line args: [] {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 26 17:49:12.707582 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 26 17:49:12.707763 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ================================================================================ {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 26 17:49:12.708017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] admin_token = **** {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.708265 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] config_dir = [] {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.708455 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.708695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] config_source = [] {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.709391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] debug = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.709594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.709959 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.710146 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.710431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.710683 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.710917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] list_limit = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.711098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] log_config_append = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.711278 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.711457 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] log_dir = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.711636 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] log_file = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.711894 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.712125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.712375 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.712623 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.712803 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.712982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.713228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.713474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.713710 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.713946 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.714184 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.714449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.714732 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.715003 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.715309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.715840 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.76/identity {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.716047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] publish_errors = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.716308 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.716565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.716766 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.717017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.717257 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.717497 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] standard_threads = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.717757 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.718024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.718214 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.718408 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] use_journal = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.718588 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] use_json = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.718766 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] use_stderr = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.718945 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] use_syslog = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.719181 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.719444 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.719686 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.719956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.720198 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.720449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.720691 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.720940 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.721181 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.721432 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.721723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.722005 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.722282 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.722559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.722822 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.723094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.723382 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.723648 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.723910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.724190 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.724443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.724684 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.724985 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.725267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.725547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.725825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.726118 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.726399 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.726674 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.726937 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.727232 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.727488 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.727726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.727962 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.728236 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.728530 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.728796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.729073 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.729311 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.729552 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.729798 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.730032 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.730286 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.730521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.730754 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.730996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.731248 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.731482 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.731723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.731957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.732198 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.732436 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.732686 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.732920 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.733160 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.733418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.733681 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.733944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.734221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.734480 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.734754 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.735028 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.735292 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.735546 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.735813 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.736054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.736287 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.736529 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.736768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.737040 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.737280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.737532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.737781 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.738023 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.738293 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.738535 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.738788 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.739031 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.739274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.739522 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.739796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.740066 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.740298 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.740539 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.740805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.741071 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.741318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.741586 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.741847 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.742094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.742359 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.742602 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.742843 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.743076 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.743309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.743563 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.743841 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.744076 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.744321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.744604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.744682 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 26 17:49:12.744857 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.744935 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 26 17:49:12.745059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] command line args: [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 26 17:49:12.745145 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.745343 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 26 17:49:12.745421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.745572 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ================================================================================ {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 26 17:49:12.745692 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.745771 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] admin_token = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.745919 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.745995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] config_dir = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.746116 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] auth.external = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.746215 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.746384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] config_source = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.746461 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.746570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] debug = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.746721 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.746807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.746957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.747050 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.747182 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.747260 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] auth.password = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.747418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] auth.token = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.747501 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.747675 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.747761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.747910 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] list_limit = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.747988 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.748092 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] log_config_append = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.748168 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.748272 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.748399 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.748478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] log_dir = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.748617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] log_file = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.748700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.748855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.748933 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.749109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.749188 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.749358 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.749442 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] credential.caching = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.749608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.749689 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.749790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.749976 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.750097 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.750303 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.750407 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.750502 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.750610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.750708 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.750838 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.750942 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.751080 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.751214 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.751357 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.751445 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.751600 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.751678 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.751843 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.751936 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.752103 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.752183 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.752396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.752618 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.76/identity {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.752710 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.752806 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] publish_errors = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.752886 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.753042 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.753124 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.753276 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.753366 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.753440 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.753609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] federation.caching = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.753689 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.753851 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.753930 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.754101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.754211 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] standard_threads = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.754375 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.754454 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.754609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.754687 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.754814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.754899 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.754974 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] use_journal = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.755091 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.755167 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] use_json = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.755282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] use_stderr = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.755355 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.756048 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] use_syslog = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.756130 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.756363 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 17:49:12.756487 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.756618 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.756706 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.756869 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.756985 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.757139 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.757218 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] identity.caching = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.757386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.757483 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.757640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.757728 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.757888 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.757975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.758129 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.758240 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.758396 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.758474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.758629 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.758715 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.758873 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.759027 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.759126 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.759205 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.759361 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.759439 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.759605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.759691 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.759841 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.759926 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.760081 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.760159 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.760328 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.760413 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.760571 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.760650 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.760818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.760896 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.761073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.761158 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.761320 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.761408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.761560 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.761644 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.761817 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.761904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.762056 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.762141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.762315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.762398 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.762557 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.762636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.762792 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.762873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.763029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.763109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.763263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.763342 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.763503 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.763581 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.763750 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.763827 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.763985 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.764062 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.764226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.764311 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.764461 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.764539 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.764697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.764774 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.764944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.765023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.765183 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.765271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.765413 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.765492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.765654 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.765732 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.765896 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.765975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.766135 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.766231 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.766400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.766481 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.766636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.766715 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.766869 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.766949 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.767102 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.767181 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.767344 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.767424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.767593 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.767671 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.767825 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.767903 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.768057 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.768136 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.768288 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.768366 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.768528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.768607 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.768767 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.768848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.769020 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.769122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.769256 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.769399 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.769485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.769637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.user = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.769707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.769925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.769993 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.770173 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.770392 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.770464 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.770657 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.770726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.770911 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.770987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.771211 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.771983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.772084 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.772219 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.772351 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.772485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.772565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.772725 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.772794 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.772959 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.773028 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.773201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.773413 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.773492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.773564 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.773701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.773781 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.773943 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.774031 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.774217 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.774344 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.774466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.774591 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.774717 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.774857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.774996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.775121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.775257 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.775339 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.775412 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.775494 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.775613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.775694 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.775853 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.775940 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.776090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.776179 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.776330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.776408 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.776573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.776658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.777087 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.777191 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.777191 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.777326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.777326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.777454 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.777516 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.777583 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.777748 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.777818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.777989 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.778064 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.778249 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.778336 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.778494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.778571 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.778733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.778801 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.778974 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.779052 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.779209 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.779277 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.779441 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.779508 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.779675 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.779745 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] resource.caching = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.779908 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.780032 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.780147 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.780228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.780381 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.780461 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.780625 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.780704 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.780864 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.780944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.781102 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.781182 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.781343 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.781421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.781581 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.781659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.781825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.781906 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.782064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] role.caching = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.782143 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.782337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] auth.external = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.782418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] role.driver = sql {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.782552 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.782633 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.782804 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.782882 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.783054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.783134 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.783296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] auth.password = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.783376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.783546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] auth.token = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.783673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.783772 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.783852 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.784017 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.784143 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.784266 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.784399 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.784499 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.784651 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.784738 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.784889 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.784975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.785143 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.785221 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.785397 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.785485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] credential.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.785650 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.785742 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.785901 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.785987 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.786148 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.786368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.786458 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.786613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.786733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.786907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.787023 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.787153 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.787488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.787663 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.787743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.787921 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.788000 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.788163 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.788245 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.788433 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.788513 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.788695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.788783 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.788943 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.789052 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.789183 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.789347 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.789426 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.789507 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.789644 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.789724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.789908 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.789987 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.790170 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.790298 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.790451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.790530 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.790687 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.790767 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.790928 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.791017 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.791166 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.791245 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.791410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] token.caching = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.791489 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.791650 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.791737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.791901 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.791980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.792140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.792218 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.792394 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.792489 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.792633 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.792713 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.792885 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.792965 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.793135 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.793225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.793374 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.793454 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.793614 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.793693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.793869 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.793957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.794103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.794182 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.794373 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.794453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.794623 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.794715 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.794875 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.794955 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.795107 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.795185 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.795344 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.795434 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.795579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.795658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.795832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.795940 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] database.connection = **** {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.796096 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.796215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.796305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.796468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.796549 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.796735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.796815 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.797008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.797105 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.797248 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.797329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.797510 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.797591 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.797743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.797830 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.797986 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.798066 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.798242 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.798326 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.798485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.798577 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.798729 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.798816 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.798974 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.799054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.799225 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.799313 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.799460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.799541 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.799710 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.799789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.799959 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.800047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.800198 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.800277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.800446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.800525 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.800619 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80260) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 26 17:49:12.800706 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.800915 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.801157 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.801473 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.801755 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.802040 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.802378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.802800 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.803052 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.803309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.803564 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.803807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.804045 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.804299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.804553 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.804794 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.805036 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.805277 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.805512 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.805756 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.806019 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.806279 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.806533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.806784 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.807034 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.807281 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.807514 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.807824 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.808040 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.808280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.808634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.808889 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.809142 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.809393 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.809643 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.809893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.810128 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.810430 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.810675 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.810917 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.811158 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.811399 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.811653 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.811904 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.812148 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.812407 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.812641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.812890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.813133 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.813373 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.813621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.813870 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.814116 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.814378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.814622 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.814868 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.815120 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.815371 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.815610 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.815862 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.816117 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.816355 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.816603 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.816845 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.817085 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.817324 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] role.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.817573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] role.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.817805 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.818625 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.818871 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.819123 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.819376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.819662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.819912 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.820164 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.820415 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.820653 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.820904 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.821156 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.821406 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.821655 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.822087 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.822353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.822609 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.822864 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.823118 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.823372 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.823618 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.823882 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.824143 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.824399 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.824636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.824872 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.825108 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.825366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.825624 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.825877 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.826118 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.826394 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.826628 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.826868 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.827108 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.827360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.827601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.827859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.828108 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.828349 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.828598 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.828845 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.829094 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.829334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.829575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.829813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.830059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.830333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.830566 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.830822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.831118 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.831380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.connection = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.831628 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.831878 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.832120 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.832364 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.832446 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:12.832606 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.832685 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:12.832825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:12.832890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.833062 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.833310 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.833625 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.834408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.834659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.834901 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.835151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.835391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.835639 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.835872 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.836115 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.836353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 17:49:12.836524 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 26 17:49:12.842367 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:12.861811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:12.862044 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:12.862301 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:12.872371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:12.872606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:12.872969 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:12.892250 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:12.892478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:12.930763 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:12.931503 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:12.984482 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:12.984724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:12.984994 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:12.985235 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.052848 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.053038 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.053115 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.053240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.053327 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.053471 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.053554 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.053697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.053789 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.053914 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.053995 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.054129 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.077332 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.077565 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.077794 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.078021 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.078103 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.078270 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.078354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.078494 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.078577 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.078715 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.078796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.078958 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.079177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.079395 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.091002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.091866 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.105359 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.105564 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.105564 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.105708 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.105781 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.105852 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.106007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.106091 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.106247 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.106331 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.106469 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.106550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.106688 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.106771 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80260) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 17:49:13.125106 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.125321 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 17:49:13.125669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.125875 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 17:49:13.125958 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.126101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 17:49:13.126182 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.126430 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.126511 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.126689 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.126772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 17:49:13.126944 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.127024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.127253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.127513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.127774 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.139340 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.139623 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.139885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.140070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.140249 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.140395 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.140606 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.140893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.148930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.149639 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.152261 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.152548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.152831 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.152968 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.153113 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.153253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.153393 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.153538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.153653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.153820 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.153923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.154104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.154332 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.154516 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.154610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.154794 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.154875 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.155071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.155151 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.155344 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.155623 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.155895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.183087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.183372 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.184868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.185119 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.185205 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.185399 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.190724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.190822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.191004 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.191089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.191285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.191367 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.198719 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.199005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.199090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.199294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.203554 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.203832 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.203950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.204235 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.204321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.204518 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.204600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.204797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.204887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.205074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.205163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.205346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.205426 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.205615 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.205697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.205884 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.225375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.225483 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.225656 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.225755 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.230398 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.230534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.234098 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.234300 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.234400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.234597 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.239580 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.239958 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.243531 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.243764 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.247358 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.247657 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.247786 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.247953 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.248120 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.248243 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.248367 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.248520 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.248666 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.248805 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.248947 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.249233 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.268475 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.268759 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.269024 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.269286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.269615 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.269925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.270224 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.270496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.277503 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.278838 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.279263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.279548 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.280560 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.280838 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.282661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.282942 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.283220 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.283500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.283776 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.284148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.284452 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.284755 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.285057 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.285359 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.298199 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.298486 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.298769 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.299063 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.299346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.299620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.299903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.300860 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.301153 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.301436 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.301724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.302007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.302413 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.302711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.318934 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.322022 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.322250 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.325353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.325596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.327276 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.328908 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.329002 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.329203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.329499 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.330667 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.332321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.332633 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.332941 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.337072 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.338740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.340613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.341996 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.342291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.342378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.342580 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.342903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.345781 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.346075 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.346421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.346699 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.352694 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.352982 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.353373 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.357276 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.357568 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.357967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.361351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.361629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.361910 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.362204 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.362483 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.362757 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.363033 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.363306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.363586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.363874 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.364145 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.364402 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.364668 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.365015 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.366023 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80260) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 17:49:13.366326 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.366609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.366889 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.367164 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.367438 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.367730 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.367995 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.368284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.368553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.368825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.369095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.369367 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.369629 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80260) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 17:49:13.404940 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55c8dffda9c0 pid: 80261 (default app) May 26 17:49:13.408274 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ad9f2-8add-4b60-8507-c4d5cd7cb4c8 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:13.408581 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ad9f2-8add-4b60-8507-c4d5cd7cb4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:13.408872 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ad9f2-8add-4b60-8507-c4d5cd7cb4c8 None None] PATH_INFO: `/v3` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:13.410661 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55c8dffda9c0 pid: 80260 (default app) May 26 17:49:13.414126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1/1] 10.222.0.76 () {50 vars in 791 bytes} [Thu May 26 17:49:13 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:21.854722 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65da85-7ce8-425d-bf30-a471e37c0c59 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:21.855307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65da85-7ce8-425d-bf30-a471e37c0c59 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:21.855307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65da85-7ce8-425d-bf30-a471e37c0c59 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:21.859942 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1/2] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:49:21.865477 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79307169-66ed-4804-b186-6c8e640c89e6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:21.865955 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79307169-66ed-4804-b186-6c8e640c89e6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:21.866126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79307169-66ed-4804-b186-6c8e640c89e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:21.936032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: INFO dbcounter [None req-79307169-66ed-4804-b186-6c8e640c89e6 None None] Registered counter for database keystone May 26 17:49:21.958941 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG oslo_db.sqlalchemy.engines [None req-79307169-66ed-4804-b186-6c8e640c89e6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80261) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 26 17:49:21.965734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writer thread running {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 26 17:49:22.154268 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG passlib.handlers.bcrypt [None req-79307169-66ed-4804-b186-6c8e640c89e6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80261) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 26 17:49:22.154694 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG passlib.handlers.bcrypt [None req-79307169-66ed-4804-b186-6c8e640c89e6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80261) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 26 17:49:22.213765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-79307169-66ed-4804-b186-6c8e640c89e6 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:22.215467 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-79307169-66ed-4804-b186-6c8e640c89e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:22.360772 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 2/3] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 496 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 26 17:49:22.529349 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65da85-7ce8-425d-bf30-a471e37c0c59 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:22.600786 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: INFO dbcounter [None req-9b65da85-7ce8-425d-bf30-a471e37c0c59 None None] Registered counter for database keystone May 26 17:49:22.620761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG oslo_db.sqlalchemy.engines [None req-9b65da85-7ce8-425d-bf30-a471e37c0c59 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80260) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 26 17:49:22.627270 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writer thread running {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 26 17:49:22.749966 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65da85-7ce8-425d-bf30-a471e37c0c59 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:22.754244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911cf33-1087-4f90-ba18-3523792e5a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:22.754910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911cf33-1087-4f90-ba18-3523792e5a93 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:22.755195 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911cf33-1087-4f90-ba18-3523792e5a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:22.755597 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911cf33-1087-4f90-ba18-3523792e5a93 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:22.765028 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911cf33-1087-4f90-ba18-3523792e5a93 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:22.819816 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c911cf33-1087-4f90-ba18-3523792e5a93 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 26 17:49:22.820057 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911cf33-1087-4f90-ba18-3523792e5a93 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:22.830565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 2/4] 10.222.0.76 () {58 vars in 1324 bytes} [Thu May 26 17:49:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 17:49:22.837779 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79307169-66ed-4804-b186-6c8e640c89e6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:22.853141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79307169-66ed-4804-b186-6c8e640c89e6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:22.857538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32c410-937b-45eb-819b-df5070db85f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:22.858733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32c410-937b-45eb-819b-df5070db85f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:22.859026 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32c410-937b-45eb-819b-df5070db85f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:22.859305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32c410-937b-45eb-819b-df5070db85f0 admin admin] PATH_INFO: `/v3/projects/5a106ee939504b208f8d5ca1deb61f2d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:22.860800 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32c410-937b-45eb-819b-df5070db85f0 admin admin] RBAC: Authorizing `identity:get_project(project_id=5a106ee939504b208f8d5ca1deb61f2d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:22.915712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7e32c410-937b-45eb-819b-df5070db85f0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 26 17:49:22.916380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32c410-937b-45eb-819b-df5070db85f0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:22.920960 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 3/5] 10.222.0.76 () {56 vars in 1207 bytes} [Thu May 26 17:49:22 2022] GET /identity/v3/projects/5a106ee939504b208f8d5ca1deb61f2d => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:25.184731 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af650475-6f18-4138-b4e0-a69563d19c51 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:25.185285 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af650475-6f18-4138-b4e0-a69563d19c51 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:25.185285 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af650475-6f18-4138-b4e0-a69563d19c51 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:25.189537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 3/6] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:49:25.196254 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a09010-3520-4bd9-ac41-01a8d7157982 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:25.196769 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a09010-3520-4bd9-ac41-01a8d7157982 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:25.196852 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a09010-3520-4bd9-ac41-01a8d7157982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:25.217470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a6a09010-3520-4bd9-ac41-01a8d7157982 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:25.219062 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a6a09010-3520-4bd9-ac41-01a8d7157982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:25.227852 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 4/7] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 26 17:49:25.250438 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af650475-6f18-4138-b4e0-a69563d19c51 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:25.263808 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af650475-6f18-4138-b4e0-a69563d19c51 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:25.268139 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582cd66-4cb3-453a-a5ee-e83fc2380730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:25.268807 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582cd66-4cb3-453a-a5ee-e83fc2380730 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:25.269095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582cd66-4cb3-453a-a5ee-e83fc2380730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:25.269379 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582cd66-4cb3-453a-a5ee-e83fc2380730 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:25.279426 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c582cd66-4cb3-453a-a5ee-e83fc2380730 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:25.281275 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c582cd66-4cb3-453a-a5ee-e83fc2380730 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:25.292284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 4/8] 10.222.0.76 () {58 vars in 1324 bytes} [Thu May 26 17:49:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 17:49:25.298264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a09010-3520-4bd9-ac41-01a8d7157982 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:25.307997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a09010-3520-4bd9-ac41-01a8d7157982 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:25.312367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc1daa-260d-4fd6-9932-667b4f9eb316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:25.313248 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc1daa-260d-4fd6-9932-667b4f9eb316 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:25.313532 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc1daa-260d-4fd6-9932-667b4f9eb316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:25.313809 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc1daa-260d-4fd6-9932-667b4f9eb316 admin admin] PATH_INFO: `/v3/users/3c37098f90fe47ab88491e90626539d9` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:25.315229 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc1daa-260d-4fd6-9932-667b4f9eb316 admin admin] RBAC: Authorizing `identity:get_user(user_id=3c37098f90fe47ab88491e90626539d9)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:25.317069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc1daa-260d-4fd6-9932-667b4f9eb316 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:25.319394 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 5/9] 10.222.0.76 () {56 vars in 1198 bytes} [Thu May 26 17:49:25 2022] GET /identity/v3/users/3c37098f90fe47ab88491e90626539d9 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:27.632271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16ad88-09d1-4a0f-8501-83217e467df7 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:27.632817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16ad88-09d1-4a0f-8501-83217e467df7 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:27.632817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16ad88-09d1-4a0f-8501-83217e467df7 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:27.634181 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 5/10] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:49:27.639442 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9898974-3b17-48c6-b448-06f3ccbba0d9 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:27.639877 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9898974-3b17-48c6-b448-06f3ccbba0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:27.640084 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9898974-3b17-48c6-b448-06f3ccbba0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:27.657625 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d9898974-3b17-48c6-b448-06f3ccbba0d9 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:27.658868 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d9898974-3b17-48c6-b448-06f3ccbba0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:27.704305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113a416-2f72-45de-bd23-cc4b8b1c3e1a None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:27.704688 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113a416-2f72-45de-bd23-cc4b8b1c3e1a None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:27.705039 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113a416-2f72-45de-bd23-cc4b8b1c3e1a None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:27.706515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 6/11] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:49:27.712801 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44ae08-b829-4716-8189-28f934cb92c8 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:27.713090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44ae08-b829-4716-8189-28f934cb92c8 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:27.713370 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44ae08-b829-4716-8189-28f934cb92c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:27.741796 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 6/12] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:27.753771 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG passlib.handlers.bcrypt [None req-dc44ae08-b829-4716-8189-28f934cb92c8 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80260) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 26 17:49:27.754221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG passlib.handlers.bcrypt [None req-dc44ae08-b829-4716-8189-28f934cb92c8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80260) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 26 17:49:27.765097 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9898974-3b17-48c6-b448-06f3ccbba0d9 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:27.790858 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:27.801591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-dc44ae08-b829-4716-8189-28f934cb92c8 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:27.804017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-dc44ae08-b829-4716-8189-28f934cb92c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:27.872784 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9898974-3b17-48c6-b448-06f3ccbba0d9 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:27.905219 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 7/13] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:49:27.909177 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e153c0-df80-401b-954b-679e70e396e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:27.910054 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e153c0-df80-401b-954b-679e70e396e3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:27.910441 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e153c0-df80-401b-954b-679e70e396e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:27.910812 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e153c0-df80-401b-954b-679e70e396e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:27.922503 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e153c0-df80-401b-954b-679e70e396e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:49:27.928630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44ae08-b829-4716-8189-28f934cb92c8 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:27.934286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-f6e153c0-df80-401b-954b-679e70e396e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:49:27.934286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:49:27.934286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:49:27.934286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:49:27.934286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:49:27.934286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:49:27.934286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:49:27.934286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:49:27.934286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:49:27.934286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:49:27.934286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:49:27.934286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:49:27.934286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:49:27.934286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:49:27.934286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:49:27.934286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:49:27.935431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:49:27.935431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:49:27.935431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:49:27.935431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:49:27.935431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:49:27.935431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:49:27.935431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:49:27.935431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:49:27.935431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:49:27.935431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:49:27.935431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:49:27.935431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:49:27.935431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:49:27.935431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:49:27.935431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:49:27.935431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:49:27.936255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:49:27.936255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:49:27.936255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:49:27.936255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:49:27.936255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:49:27.936255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:49:27.936255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:49:27.936255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:49:27.936255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:49:27.936255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:49:27.936255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:49:27.936255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:49:27.936255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 7/14] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:49:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:49:27.941306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e153c0-df80-401b-954b-679e70e396e3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:28.144720 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:28.145991 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44ae08-b829-4716-8189-28f934cb92c8 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:28.149006 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:28.165509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e153c0-df80-401b-954b-679e70e396e3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:28.187055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168fb22-cb86-4844-a92e-d38f8693311f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:28.187966 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168fb22-cb86-4844-a92e-d38f8693311f None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:28.188250 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168fb22-cb86-4844-a92e-d38f8693311f None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:28.188528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168fb22-cb86-4844-a92e-d38f8693311f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:28.201693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad48a2a-c2f9-4bb9-8bc6-b882dec0773f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:28.202402 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad48a2a-c2f9-4bb9-8bc6-b882dec0773f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:28.202734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad48a2a-c2f9-4bb9-8bc6-b882dec0773f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:28.203007 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad48a2a-c2f9-4bb9-8bc6-b882dec0773f None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:28.203769 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad48a2a-c2f9-4bb9-8bc6-b882dec0773f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:28.204083 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad48a2a-c2f9-4bb9-8bc6-b882dec0773f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:28.205564 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad48a2a-c2f9-4bb9-8bc6-b882dec0773f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:28.217981 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 8/15] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:49:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:49:28.223485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad48a2a-c2f9-4bb9-8bc6-b882dec0773f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:28.258177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168fb22-cb86-4844-a92e-d38f8693311f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:28.259732 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168fb22-cb86-4844-a92e-d38f8693311f None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:28.326905 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad48a2a-c2f9-4bb9-8bc6-b882dec0773f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:28.363260 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 8/16] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:49:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 17:49:28.366328 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb36118-5afd-4f23-80aa-18c220f435fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:28.367223 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb36118-5afd-4f23-80aa-18c220f435fa None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:28.367505 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb36118-5afd-4f23-80aa-18c220f435fa None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:28.367777 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb36118-5afd-4f23-80aa-18c220f435fa None admin] PATH_INFO: `/v3/users/3c37098f90fe47ab88491e90626539d9` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:28.369382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb36118-5afd-4f23-80aa-18c220f435fa None admin] RBAC: Authorizing `identity:get_user(user_id=3c37098f90fe47ab88491e90626539d9)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:28.369598 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168fb22-cb86-4844-a92e-d38f8693311f None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:28.370908 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb36118-5afd-4f23-80aa-18c220f435fa None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:28.372613 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 9/17] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:49:28 2022] GET /identity/v3/users/3c37098f90fe47ab88491e90626539d9 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:49:28.372758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:28.378162 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb36118-5afd-4f23-80aa-18c220f435fa None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:28.393281 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:28.473715 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168fb22-cb86-4844-a92e-d38f8693311f None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:28.480534 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb36118-5afd-4f23-80aa-18c220f435fa None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:28.514438 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b58f6c-fffa-4bfe-b1cc-95df6030b10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:28.515385 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b58f6c-fffa-4bfe-b1cc-95df6030b10d None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:28.515555 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b58f6c-fffa-4bfe-b1cc-95df6030b10d None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:28.515786 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b58f6c-fffa-4bfe-b1cc-95df6030b10d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:28.520342 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e8084e-3cb7-4092-8685-dcf4bf49fc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:28.521346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e8084e-3cb7-4092-8685-dcf4bf49fc3b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:28.521806 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e8084e-3cb7-4092-8685-dcf4bf49fc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:28.522114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e8084e-3cb7-4092-8685-dcf4bf49fc3b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:28.523566 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e8084e-3cb7-4092-8685-dcf4bf49fc3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:28.525152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e8084e-3cb7-4092-8685-dcf4bf49fc3b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:28.526854 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 10/18] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:49:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:49:28.533265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e8084e-3cb7-4092-8685-dcf4bf49fc3b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:28.538263 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b58f6c-fffa-4bfe-b1cc-95df6030b10d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:28.539782 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b58f6c-fffa-4bfe-b1cc-95df6030b10d None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:28.562638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-95b58f6c-fffa-4bfe-b1cc-95df6030b10d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:49:28.562638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:49:28.562638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:49:28.562638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:49:28.562638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:49:28.562638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:49:28.562638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:49:28.562638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:49:28.562638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:49:28.562638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:49:28.562638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:49:28.562638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:49:28.562638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:49:28.562638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:49:28.562638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:49:28.563814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:49:28.563814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:49:28.563814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:49:28.563814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:49:28.563814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:49:28.563814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:49:28.563814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:49:28.563814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:49:28.563814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:49:28.563814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:49:28.563814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:49:28.563814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:49:28.563814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:49:28.563814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:49:28.563814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:49:28.563814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:49:28.563814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:49:28.564768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:49:28.564768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:49:28.564768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:49:28.564768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:49:28.564768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 9/19] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:49:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:49:28.570030 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b58f6c-fffa-4bfe-b1cc-95df6030b10d None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:28.622337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:28.629514 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:28.643117 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e8084e-3cb7-4092-8685-dcf4bf49fc3b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:28.675762 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b58f6c-fffa-4bfe-b1cc-95df6030b10d None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:28.678750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18d187-bbdd-4b25-95d1-94a80c0ded0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:28.679415 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18d187-bbdd-4b25-95d1-94a80c0ded0e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:28.679702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18d187-bbdd-4b25-95d1-94a80c0ded0e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:28.679981 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18d187-bbdd-4b25-95d1-94a80c0ded0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:28.680824 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18d187-bbdd-4b25-95d1-94a80c0ded0e None admin] RBAC: Adding query filter params (role.id=4fa8f06085c741a5b3cf64e63cc0754f, scope.domain.id=default, user.id=3c37098f90fe47ab88491e90626539d9) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:28.681136 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18d187-bbdd-4b25-95d1-94a80c0ded0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:28.682616 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18d187-bbdd-4b25-95d1-94a80c0ded0e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:28.689598 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 11/20] 10.222.0.76 () {56 vars in 1322 bytes} [Thu May 26 17:49:28 2022] GET /identity/v3/role_assignments?user.id=3c37098f90fe47ab88491e90626539d9&scope.domain.id=default&role.id=4fa8f06085c741a5b3cf64e63cc0754f => generated 233 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:49:28.713640 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b280e-6272-4de6-920c-7b0b41a3b5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:28.714473 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b280e-6272-4de6-920c-7b0b41a3b5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:28.714765 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b280e-6272-4de6-920c-7b0b41a3b5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:28.715047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b280e-6272-4de6-920c-7b0b41a3b5d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:28.715816 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b280e-6272-4de6-920c-7b0b41a3b5d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:28.716126 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b280e-6272-4de6-920c-7b0b41a3b5d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:28.717594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b280e-6272-4de6-920c-7b0b41a3b5d8 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:28.738364 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 10/21] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:49:31.294665 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea37792-a27f-4d05-a4b0-4c3bffdf8e44 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:31.295200 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea37792-a27f-4d05-a4b0-4c3bffdf8e44 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:31.295282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea37792-a27f-4d05-a4b0-4c3bffdf8e44 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:31.296952 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 12/22] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:31.308453 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b53053-f74a-4ec3-82a1-5901dc285d76 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:31.309371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b53053-f74a-4ec3-82a1-5901dc285d76 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:31.309830 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b53053-f74a-4ec3-82a1-5901dc285d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:31.339291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-23b53053-f74a-4ec3-82a1-5901dc285d76 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:31.340813 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-23b53053-f74a-4ec3-82a1-5901dc285d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:31.443763 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 11/23] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:31.480901 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea37792-a27f-4d05-a4b0-4c3bffdf8e44 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:31.595294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5eafd0-a475-451d-ba66-89199a3c63d7 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:31.595893 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5eafd0-a475-451d-ba66-89199a3c63d7 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:31.596354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5eafd0-a475-451d-ba66-89199a3c63d7 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:31.597961 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 12/24] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:49:31.623259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878568da-8757-481b-a319-b87f07f07b26 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:31.623851 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878568da-8757-481b-a319-b87f07f07b26 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:31.624314 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878568da-8757-481b-a319-b87f07f07b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:31.630709 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea37792-a27f-4d05-a4b0-4c3bffdf8e44 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:31.666940 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-878568da-8757-481b-a319-b87f07f07b26 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:31.668514 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-878568da-8757-481b-a319-b87f07f07b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:31.681591 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5029d4-364b-4e01-8562-b3d1acb0a859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:31.682748 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5029d4-364b-4e01-8562-b3d1acb0a859 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:31.683217 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5029d4-364b-4e01-8562-b3d1acb0a859 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:31.683670 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5029d4-364b-4e01-8562-b3d1acb0a859 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:31.710778 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5029d4-364b-4e01-8562-b3d1acb0a859 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:49:31.718990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-dd5029d4-364b-4e01-8562-b3d1acb0a859 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:49:31.718990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:49:31.718990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:49:31.718990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:49:31.718990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:49:31.718990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:49:31.718990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:49:31.718990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:49:31.718990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:49:31.718990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:49:31.718990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:49:31.718990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:49:31.718990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:49:31.718990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:49:31.718990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:49:31.718990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:49:31.719863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:49:31.719863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:49:31.719863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:49:31.719863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:49:31.719863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:49:31.719863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:49:31.719863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:49:31.719863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:49:31.719863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:49:31.719863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:49:31.719863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:49:31.719863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:49:31.719863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:49:31.719863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:49:31.719863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:49:31.719863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:49:31.720608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:49:31.720608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:49:31.720608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:49:31.720608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:49:31.720608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:49:31.720608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:49:31.720608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:49:31.720608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:49:31.720608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:49:31.720608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:49:31.720608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:49:31.720608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:49:31.730755 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 13/25] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:49:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:49:31.740340 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0d9c6-7357-45c5-84c2-6e79f6fd04c8 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:31.741700 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0d9c6-7357-45c5-84c2-6e79f6fd04c8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:31.742696 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0d9c6-7357-45c5-84c2-6e79f6fd04c8 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:31.744721 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 14/26] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:31 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:31.750222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09678c3-92c3-427e-89ac-5eb265dfb4f4 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:31.750712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09678c3-92c3-427e-89ac-5eb265dfb4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:31.751138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09678c3-92c3-427e-89ac-5eb265dfb4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:31.763781 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:31.765256 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:31.781617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a09678c3-92c3-427e-89ac-5eb265dfb4f4 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:31.782951 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a09678c3-92c3-427e-89ac-5eb265dfb4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:31.808770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 13/27] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:49:31.809729 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878568da-8757-481b-a319-b87f07f07b26 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:31.883462 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 15/28] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:31.884707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09678c3-92c3-427e-89ac-5eb265dfb4f4 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:32.002598 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09678c3-92c3-427e-89ac-5eb265dfb4f4 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:32.041889 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ddaad-8ec0-4317-b605-eefebd6d8618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:32.043025 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ddaad-8ec0-4317-b605-eefebd6d8618 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:32.043507 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ddaad-8ec0-4317-b605-eefebd6d8618 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:32.044038 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ddaad-8ec0-4317-b605-eefebd6d8618 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:32.044982 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ddaad-8ec0-4317-b605-eefebd6d8618 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:32.046657 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ddaad-8ec0-4317-b605-eefebd6d8618 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:32.054565 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:32.078708 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878568da-8757-481b-a319-b87f07f07b26 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:32.151625 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae3225-8728-46f6-bc79-6b227e60c0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:32.152323 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae3225-8728-46f6-bc79-6b227e60c0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:32.152609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae3225-8728-46f6-bc79-6b227e60c0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:32.152885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae3225-8728-46f6-bc79-6b227e60c0f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:32.153655 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae3225-8728-46f6-bc79-6b227e60c0f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:32.153961 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae3225-8728-46f6-bc79-6b227e60c0f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:32.155402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae3225-8728-46f6-bc79-6b227e60c0f2 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:32.174512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 14/29] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:49:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:32.174512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae3225-8728-46f6-bc79-6b227e60c0f2 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:32.238631 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:32.255475 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 16/30] 10.222.0.76 () {60 vars in 1131 bytes} [Thu May 26 17:49:31 2022] POST /identity/v3/roles => generated 219 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:49:32.263366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ddaad-8ec0-4317-b605-eefebd6d8618 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:32.307505 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae3225-8728-46f6-bc79-6b227e60c0f2 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:32.354219 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1874f4c-fc0c-4fcb-8225-42cc83f45730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:32.354219 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1874f4c-fc0c-4fcb-8225-42cc83f45730 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:32.354219 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1874f4c-fc0c-4fcb-8225-42cc83f45730 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:32.354219 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1874f4c-fc0c-4fcb-8225-42cc83f45730 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:32.374221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1874f4c-fc0c-4fcb-8225-42cc83f45730 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:32.374221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1874f4c-fc0c-4fcb-8225-42cc83f45730 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:32.387307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-b1874f4c-fc0c-4fcb-8225-42cc83f45730 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:49:32.387307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:49:32.387307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:49:32.387307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:49:32.387307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:49:32.387307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:49:32.387307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:49:32.387307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:49:32.387307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:49:32.387307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:49:32.387307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:49:32.387307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:49:32.387307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:49:32.387307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:49:32.387307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:49:32.388179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:49:32.388179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:49:32.388179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:49:32.388179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:49:32.388179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:49:32.388179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:49:32.388179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:49:32.388179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:49:32.388179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:49:32.388179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:49:32.388179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:49:32.388179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:49:32.388179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:49:32.388179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:49:32.388179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:49:32.388179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:49:32.388179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:49:32.388981 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:49:32.388981 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:49:32.388981 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:49:32.388981 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:49:32.388981 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 15/31] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:49:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:49:32.390509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29127282-0af6-4824-a8ce-2715f89c5ee7 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:32.390634 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29127282-0af6-4824-a8ce-2715f89c5ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:32.392458 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29127282-0af6-4824-a8ce-2715f89c5ee7 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:32.392458 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 16/32] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:32.395083 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29127282-0af6-4824-a8ce-2715f89c5ee7 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:32.506852 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:32.517248 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29127282-0af6-4824-a8ce-2715f89c5ee7 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:32.533623 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ddaad-8ec0-4317-b605-eefebd6d8618 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:32.578665 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ad027-328d-4ee9-86fc-7d6c800ff47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:32.580071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ad027-328d-4ee9-86fc-7d6c800ff47e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:32.580071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ad027-328d-4ee9-86fc-7d6c800ff47e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:32.580349 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ad027-328d-4ee9-86fc-7d6c800ff47e None admin] PATH_INFO: `/v3/users/3c37098f90fe47ab88491e90626539d9` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:32.581825 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ad027-328d-4ee9-86fc-7d6c800ff47e None admin] RBAC: Authorizing `identity:get_user(user_id=3c37098f90fe47ab88491e90626539d9)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:32.583140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5840f5-b054-477f-ae74-dbcd293ffe14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:32.583765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ad027-328d-4ee9-86fc-7d6c800ff47e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:32.584269 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5840f5-b054-477f-ae74-dbcd293ffe14 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:32.584744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5840f5-b054-477f-ae74-dbcd293ffe14 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:32.585209 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5840f5-b054-477f-ae74-dbcd293ffe14 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:32.585775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 17/33] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:49:32 2022] GET /identity/v3/users/3c37098f90fe47ab88491e90626539d9 => generated 247 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:32.586688 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5840f5-b054-477f-ae74-dbcd293ffe14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:32.587192 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5840f5-b054-477f-ae74-dbcd293ffe14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:32.588789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5840f5-b054-477f-ae74-dbcd293ffe14 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:32.593556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ddf1e-0eaf-4c77-83b7-a15e9c31be0a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:32.593860 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ddf1e-0eaf-4c77-83b7-a15e9c31be0a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:32.594148 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ddf1e-0eaf-4c77-83b7-a15e9c31be0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:32.605512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 17/34] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:32.606741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5840f5-b054-477f-ae74-dbcd293ffe14 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:32.619283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2c0ddf1e-0eaf-4c77-83b7-a15e9c31be0a None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:32.621133 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2c0ddf1e-0eaf-4c77-83b7-a15e9c31be0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:32.692470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:32.706581 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 18/35] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:32.711240 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ddf1e-0eaf-4c77-83b7-a15e9c31be0a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:32.744801 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5840f5-b054-477f-ae74-dbcd293ffe14 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:32.788568 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43f68d-e096-4501-9ca4-4bef022d7770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:32.789337 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43f68d-e096-4501-9ca4-4bef022d7770 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:32.789603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43f68d-e096-4501-9ca4-4bef022d7770 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:32.789882 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43f68d-e096-4501-9ca4-4bef022d7770 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:32.791540 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43f68d-e096-4501-9ca4-4bef022d7770 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:32.793000 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43f68d-e096-4501-9ca4-4bef022d7770 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:32.794708 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 18/36] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:49:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:32.800464 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099f09d-38be-4810-8d7a-4143124351ee None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:32.800761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099f09d-38be-4810-8d7a-4143124351ee None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:32.801048 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099f09d-38be-4810-8d7a-4143124351ee None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:32.806377 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 19/37] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:32.806377 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099f09d-38be-4810-8d7a-4143124351ee None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:32.830325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:32.839581 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ddf1e-0eaf-4c77-83b7-a15e9c31be0a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:32.879719 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdad1a-de6f-4861-88c3-14443898df09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:32.880480 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdad1a-de6f-4861-88c3-14443898df09 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:32.880774 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdad1a-de6f-4861-88c3-14443898df09 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:32.881064 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdad1a-de6f-4861-88c3-14443898df09 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:32.881785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdad1a-de6f-4861-88c3-14443898df09 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:32.883362 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdad1a-de6f-4861-88c3-14443898df09 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:32.915065 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 19/38] 10.222.0.76 () {60 vars in 1141 bytes} [Thu May 26 17:49:32 2022] POST /identity/v3/projects => generated 301 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:49:32.915934 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdad1a-de6f-4861-88c3-14443898df09 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:32.933253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099f09d-38be-4810-8d7a-4143124351ee None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:32.979572 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071f447-41fd-4584-8fee-f892f730679a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:32.980347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071f447-41fd-4584-8fee-f892f730679a None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:32.980632 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071f447-41fd-4584-8fee-f892f730679a None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:32.980908 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071f447-41fd-4584-8fee-f892f730679a None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:32.981707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1071f447-41fd-4584-8fee-f892f730679a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:32.983528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1071f447-41fd-4584-8fee-f892f730679a None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:32.986620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:33.063211 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdad1a-de6f-4861-88c3-14443898df09 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:33.109280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e16a6a-99a1-4339-9d2a-2e48be08ebe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:33.110651 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e16a6a-99a1-4339-9d2a-2e48be08ebe1 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:33.111127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e16a6a-99a1-4339-9d2a-2e48be08ebe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:33.111630 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e16a6a-99a1-4339-9d2a-2e48be08ebe1 None admin] PATH_INFO: `/v3/domains/default/users/3c37098f90fe47ab88491e90626539d9/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:33.114438 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e16a6a-99a1-4339-9d2a-2e48be08ebe1 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3c37098f90fe47ab88491e90626539d9, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:33.116677 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e16a6a-99a1-4339-9d2a-2e48be08ebe1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:33.120580 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 20/39] 10.222.0.76 () {60 vars in 1131 bytes} [Thu May 26 17:49:32 2022] POST /identity/v3/roles => generated 225 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:49:33.122458 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748e2c1-1eda-41dd-a07c-0b69702f001b None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:33.122757 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748e2c1-1eda-41dd-a07c-0b69702f001b None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:33.123165 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748e2c1-1eda-41dd-a07c-0b69702f001b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:33.145181 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-e748e2c1-1eda-41dd-a07c-0b69702f001b None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:33.146612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 20/40] 10.222.0.76 () {58 vars in 1361 bytes} [Thu May 26 17:49:32 2022] PUT /identity/v3/domains/default/users/3c37098f90fe47ab88491e90626539d9/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:49:33.147920 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-e748e2c1-1eda-41dd-a07c-0b69702f001b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:33.242185 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 21/41] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:33.265596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5288012-021b-4b77-b172-05247b52a9af None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:33.265939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5288012-021b-4b77-b172-05247b52a9af None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:33.266174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5288012-021b-4b77-b172-05247b52a9af None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:33.268010 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 21/42] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:33.268818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748e2c1-1eda-41dd-a07c-0b69702f001b None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:33.274768 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354992bd-73aa-4a13-ad53-a890ee03267f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:33.275093 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354992bd-73aa-4a13-ad53-a890ee03267f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:33.275345 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354992bd-73aa-4a13-ad53-a890ee03267f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:33.282423 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:33.301231 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-354992bd-73aa-4a13-ad53-a890ee03267f None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:33.302817 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-354992bd-73aa-4a13-ad53-a890ee03267f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:33.413317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 22/43] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:49:33.414804 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748e2c1-1eda-41dd-a07c-0b69702f001b None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:33.437508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354992bd-73aa-4a13-ad53-a890ee03267f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:33.459351 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59357f-afae-4c25-8b80-e53e9c36f471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:33.460793 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59357f-afae-4c25-8b80-e53e9c36f471 None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:33.460793 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59357f-afae-4c25-8b80-e53e9c36f471 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:33.460793 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59357f-afae-4c25-8b80-e53e9c36f471 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:33.461347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a59357f-afae-4c25-8b80-e53e9c36f471 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:33.463000 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a59357f-afae-4c25-8b80-e53e9c36f471 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:33.482068 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 22/44] 10.222.0.76 () {60 vars in 1131 bytes} [Thu May 26 17:49:33 2022] POST /identity/v3/roles => generated 223 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:49:33.489260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:33.573101 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354992bd-73aa-4a13-ad53-a890ee03267f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:33.613114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2699f-d088-4ee7-8e48-f26021b085e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:33.613980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2699f-d088-4ee7-8e48-f26021b085e1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:33.614613 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2699f-d088-4ee7-8e48-f26021b085e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:33.614613 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2699f-d088-4ee7-8e48-f26021b085e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:33.615897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2699f-d088-4ee7-8e48-f26021b085e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:33.617407 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2699f-d088-4ee7-8e48-f26021b085e1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:33.619390 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 23/45] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:49:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:49:33.624826 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59357f-afae-4c25-8b80-e53e9c36f471 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:33.702151 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:33.747062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59357f-afae-4c25-8b80-e53e9c36f471 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:33.783279 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edee891-3288-4344-a836-5d7638909a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:33.783951 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edee891-3288-4344-a836-5d7638909a59 None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:33.784236 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edee891-3288-4344-a836-5d7638909a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:33.784512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edee891-3288-4344-a836-5d7638909a59 None admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:33.785235 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edee891-3288-4344-a836-5d7638909a59 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:33.787381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edee891-3288-4344-a836-5d7638909a59 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:33.814213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 23/46] 10.222.0.76 () {60 vars in 1141 bytes} [Thu May 26 17:49:33 2022] POST /identity/v3/projects => generated 312 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:49:35.502542 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f52a5-a138-4705-b196-3ecbe7180356 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:35.503104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f52a5-a138-4705-b196-3ecbe7180356 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:35.503187 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f52a5-a138-4705-b196-3ecbe7180356 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:35.504668 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 24/47] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:35.510547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973d9d9-63bf-4dd6-91fc-5b8af7881873 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:35.510989 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973d9d9-63bf-4dd6-91fc-5b8af7881873 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:35.511265 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973d9d9-63bf-4dd6-91fc-5b8af7881873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:35.530010 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-8973d9d9-63bf-4dd6-91fc-5b8af7881873 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:35.531279 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-8973d9d9-63bf-4dd6-91fc-5b8af7881873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:35.605192 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 24/48] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:35.628229 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f52a5-a138-4705-b196-3ecbe7180356 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:35.737415 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f52a5-a138-4705-b196-3ecbe7180356 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:35.739002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:35.774867 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3b559-44f5-4630-8c13-7d7e301fa091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:35.775590 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3b559-44f5-4630-8c13-7d7e301fa091 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:35.775884 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3b559-44f5-4630-8c13-7d7e301fa091 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:35.776151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3b559-44f5-4630-8c13-7d7e301fa091 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:35.787790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3b559-44f5-4630-8c13-7d7e301fa091 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:49:35.795423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-4ea3b559-44f5-4630-8c13-7d7e301fa091 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:49:35.795423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:49:35.795423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:49:35.795423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:49:35.795423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:49:35.795423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:49:35.795423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:49:35.795423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:49:35.795423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:49:35.795423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:49:35.795423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:49:35.795423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:49:35.795423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:49:35.795423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:49:35.795423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:49:35.795423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:49:35.796428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:49:35.796428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:49:35.796428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:49:35.796428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:49:35.796428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:49:35.796428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:49:35.796428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:49:35.796428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:49:35.796428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:49:35.796428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:49:35.796428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:49:35.796428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:49:35.796428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:49:35.796428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:49:35.796428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:49:35.796428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:49:35.797228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:49:35.797228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:49:35.797228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:49:35.797228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:49:35.797228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:49:35.797228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:49:35.797228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:49:35.797228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:49:35.797228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:49:35.797228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:49:35.797228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:49:35.797228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:49:35.797228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 25/49] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:49:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:49:35.802814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973d9d9-63bf-4dd6-91fc-5b8af7881873 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:35.855229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:35.910316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973d9d9-63bf-4dd6-91fc-5b8af7881873 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:35.945737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de82d6-a173-4314-8735-2edae3bd729b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:35.946447 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de82d6-a173-4314-8735-2edae3bd729b None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:35.946732 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de82d6-a173-4314-8735-2edae3bd729b None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:35.947005 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de82d6-a173-4314-8735-2edae3bd729b None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:35.947757 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de82d6-a173-4314-8735-2edae3bd729b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:35.948064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de82d6-a173-4314-8735-2edae3bd729b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:35.949446 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de82d6-a173-4314-8735-2edae3bd729b None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:35.957808 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 25/50] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:49:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:35.963874 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3b559-44f5-4630-8c13-7d7e301fa091 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:36.048601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9359134-bd52-4e30-b7d4-7cd98f08f60e None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:36.048811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9359134-bd52-4e30-b7d4-7cd98f08f60e None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:36.049092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9359134-bd52-4e30-b7d4-7cd98f08f60e None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:36.050527 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 26/51] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:49:36.056287 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf5782-3f45-4371-b857-6aef3f07ea66 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:36.056578 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf5782-3f45-4371-b857-6aef3f07ea66 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:36.056863 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf5782-3f45-4371-b857-6aef3f07ea66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:36.069239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3b559-44f5-4630-8c13-7d7e301fa091 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:36.074768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-d1bf5782-3f45-4371-b857-6aef3f07ea66 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:36.075990 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-d1bf5782-3f45-4371-b857-6aef3f07ea66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:36.103826 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca79dc-0fe4-42ec-a78c-fae97246a45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:36.104928 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca79dc-0fe4-42ec-a78c-fae97246a45b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:36.105215 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca79dc-0fe4-42ec-a78c-fae97246a45b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:36.105498 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca79dc-0fe4-42ec-a78c-fae97246a45b None admin] PATH_INFO: `/v3/users/3c37098f90fe47ab88491e90626539d9` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:36.106823 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca79dc-0fe4-42ec-a78c-fae97246a45b None admin] RBAC: Authorizing `identity:get_user(user_id=3c37098f90fe47ab88491e90626539d9)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:36.108294 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca79dc-0fe4-42ec-a78c-fae97246a45b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:36.110100 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 26/52] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:49:35 2022] GET /identity/v3/users/3c37098f90fe47ab88491e90626539d9 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:36.116220 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca79dc-0fe4-42ec-a78c-fae97246a45b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:36.148059 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 27/53] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:49:36.149245 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:36.171179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf5782-3f45-4371-b857-6aef3f07ea66 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:36.215804 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:36.221543 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca79dc-0fe4-42ec-a78c-fae97246a45b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:36.258457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f65e99-eebd-4c38-ae11-c4c86480873e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:36.259254 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f65e99-eebd-4c38-ae11-c4c86480873e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:36.259561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f65e99-eebd-4c38-ae11-c4c86480873e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:36.259857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f65e99-eebd-4c38-ae11-c4c86480873e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:36.261032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f65e99-eebd-4c38-ae11-c4c86480873e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:36.262525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f65e99-eebd-4c38-ae11-c4c86480873e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:36.264177 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 27/54] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:49:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:49:36.269890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f65e99-eebd-4c38-ae11-c4c86480873e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:36.273082 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf5782-3f45-4371-b857-6aef3f07ea66 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:36.309341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396cb44-9a21-48e7-ab8c-08b1754a99f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:36.310076 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396cb44-9a21-48e7-ab8c-08b1754a99f8 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:36.310389 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396cb44-9a21-48e7-ab8c-08b1754a99f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:36.310663 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396cb44-9a21-48e7-ab8c-08b1754a99f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:36.311906 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396cb44-9a21-48e7-ab8c-08b1754a99f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:36.313394 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396cb44-9a21-48e7-ab8c-08b1754a99f8 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:36.315157 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 28/55] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:49:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:49:36.321767 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396cb44-9a21-48e7-ab8c-08b1754a99f8 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:36.374389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:36.378391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f65e99-eebd-4c38-ae11-c4c86480873e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:36.415308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6ec0a-3672-4c39-97bc-d7ccc3524286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:36.416080 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6ec0a-3672-4c39-97bc-d7ccc3524286 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:36.416368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6ec0a-3672-4c39-97bc-d7ccc3524286 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:36.416646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6ec0a-3672-4c39-97bc-d7ccc3524286 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:36.417481 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6ec0a-3672-4c39-97bc-d7ccc3524286 None admin] RBAC: Adding query filter params (role.id=4fa8f06085c741a5b3cf64e63cc0754f, scope.domain.id=default, user.id=3c37098f90fe47ab88491e90626539d9) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:36.417796 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6ec0a-3672-4c39-97bc-d7ccc3524286 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:36.419296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6ec0a-3672-4c39-97bc-d7ccc3524286 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:36.425074 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 28/56] 10.222.0.76 () {56 vars in 1322 bytes} [Thu May 26 17:49:36 2022] GET /identity/v3/role_assignments?user.id=3c37098f90fe47ab88491e90626539d9&scope.domain.id=default&role.id=4fa8f06085c741a5b3cf64e63cc0754f => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:49:36.428418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396cb44-9a21-48e7-ab8c-08b1754a99f8 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:36.440745 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:36.469370 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74095b8f-d118-4755-afbe-0f8ec3188395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:36.470054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74095b8f-d118-4755-afbe-0f8ec3188395 None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:36.470373 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74095b8f-d118-4755-afbe-0f8ec3188395 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:36.470654 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74095b8f-d118-4755-afbe-0f8ec3188395 None admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:36.471394 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74095b8f-d118-4755-afbe-0f8ec3188395 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:36.472860 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74095b8f-d118-4755-afbe-0f8ec3188395 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:36.503072 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 29/57] 10.222.0.76 () {60 vars in 1140 bytes} [Thu May 26 17:49:36 2022] POST /identity/v3/projects => generated 298 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:49:38.765476 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96850e0d-b1a2-40ac-89fd-ed07a9773e72 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:38.765994 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96850e0d-b1a2-40ac-89fd-ed07a9773e72 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:38.766085 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96850e0d-b1a2-40ac-89fd-ed07a9773e72 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:38.767450 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 29/58] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:38.773026 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e5bae-0faa-4eeb-9c26-ed35823a23d2 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:38.773509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e5bae-0faa-4eeb-9c26-ed35823a23d2 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:38.773612 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e5bae-0faa-4eeb-9c26-ed35823a23d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:38.792214 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-153e5bae-0faa-4eeb-9c26-ed35823a23d2 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:38.793639 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-153e5bae-0faa-4eeb-9c26-ed35823a23d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:38.865077 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 30/59] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:38.889052 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96850e0d-b1a2-40ac-89fd-ed07a9773e72 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:38.990472 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96850e0d-b1a2-40ac-89fd-ed07a9773e72 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:39.027602 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ad9df-09ec-46bf-ab81-65352fba2ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:39.028328 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ad9df-09ec-46bf-ab81-65352fba2ecf None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:39.028618 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ad9df-09ec-46bf-ab81-65352fba2ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:39.028900 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ad9df-09ec-46bf-ab81-65352fba2ecf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:39.030149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ad9df-09ec-46bf-ab81-65352fba2ecf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:39.031682 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ad9df-09ec-46bf-ab81-65352fba2ecf None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:39.033392 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 30/60] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:49:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:39.039887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e5bae-0faa-4eeb-9c26-ed35823a23d2 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:39.131240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:39.141238 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e5bae-0faa-4eeb-9c26-ed35823a23d2 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:39.176123 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47794893-afbb-45ef-b384-311ff1115743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:39.177387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47794893-afbb-45ef-b384-311ff1115743 None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:39.177702 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47794893-afbb-45ef-b384-311ff1115743 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:39.178128 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47794893-afbb-45ef-b384-311ff1115743 None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:39.178887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47794893-afbb-45ef-b384-311ff1115743 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:39.180393 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47794893-afbb-45ef-b384-311ff1115743 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:39.205445 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 31/61] 10.222.0.76 () {60 vars in 1132 bytes} [Thu May 26 17:49:39 2022] POST /identity/v3/users => generated 275 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:49:42.497702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1848b4-2257-4082-8a1b-65c102716032 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:42.498749 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1848b4-2257-4082-8a1b-65c102716032 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:42.499241 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1848b4-2257-4082-8a1b-65c102716032 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:42.500984 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 31/62] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:42.508115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bde83e-c0b1-4c87-97f8-9c3809a73571 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:42.509068 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bde83e-c0b1-4c87-97f8-9c3809a73571 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:42.509550 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bde83e-c0b1-4c87-97f8-9c3809a73571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:42.534461 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-81bde83e-c0b1-4c87-97f8-9c3809a73571 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:42.536554 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-81bde83e-c0b1-4c87-97f8-9c3809a73571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:42.629456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 32/63] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:42.666165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1848b4-2257-4082-8a1b-65c102716032 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:42.689232 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c76893-c1ec-422b-9225-40c348e631a2 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:42.689542 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c76893-c1ec-422b-9225-40c348e631a2 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:42.689813 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c76893-c1ec-422b-9225-40c348e631a2 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:42.691339 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 33/64] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:49:42.696930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198a5a0-3f3c-4de7-9dc5-255ddf24ee74 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:42.697497 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198a5a0-3f3c-4de7-9dc5-255ddf24ee74 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:42.697980 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198a5a0-3f3c-4de7-9dc5-255ddf24ee74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:42.724120 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-0198a5a0-3f3c-4de7-9dc5-255ddf24ee74 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:42.726585 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:42.727019 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-0198a5a0-3f3c-4de7-9dc5-255ddf24ee74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:42.868741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 34/65] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:49:42.876392 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fbf37-0dfe-455a-9cbd-c80bc51647cc None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:42.876691 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fbf37-0dfe-455a-9cbd-c80bc51647cc None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:42.876976 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fbf37-0dfe-455a-9cbd-c80bc51647cc None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:42.878357 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1848b4-2257-4082-8a1b-65c102716032 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:42.878473 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 35/66] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:49:42.886476 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d98538-c03b-4ecd-8f5f-4f66b23dd3e0 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:42.886768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d98538-c03b-4ecd-8f5f-4f66b23dd3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:42.887052 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d98538-c03b-4ecd-8f5f-4f66b23dd3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:42.912788 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-56d98538-c03b-4ecd-8f5f-4f66b23dd3e0 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:42.914304 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-56d98538-c03b-4ecd-8f5f-4f66b23dd3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:42.916723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:42.936130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a72a9-7883-44d8-aead-fd427c9dc36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:42.936938 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a72a9-7883-44d8-aead-fd427c9dc36e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:42.937233 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a72a9-7883-44d8-aead-fd427c9dc36e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:42.937515 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a72a9-7883-44d8-aead-fd427c9dc36e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:42.949667 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a72a9-7883-44d8-aead-fd427c9dc36e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:49:42.958873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-404a72a9-7883-44d8-aead-fd427c9dc36e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:49:42.958873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:49:42.958873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:49:42.958873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:49:42.958873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:49:42.958873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:49:42.958873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:49:42.958873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:49:42.958873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:49:42.958873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:49:42.958873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:49:42.958873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:49:42.958873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:49:42.958873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:49:42.958873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:49:42.958873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:49:42.959857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:49:42.959857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:49:42.959857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:49:42.959857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:49:42.959857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:49:42.959857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:49:42.959857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:49:42.959857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:49:42.959857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:49:42.959857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:49:42.959857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:49:42.959857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:49:42.959857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:49:42.959857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:49:42.959857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:49:42.959857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:49:42.960614 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:49:42.960614 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:49:42.960614 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:49:42.960614 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:49:42.960614 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:49:42.960614 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:49:42.960614 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:49:42.960614 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:49:42.960614 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:49:42.960614 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:49:42.960614 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:49:42.960614 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:49:42.963386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 32/67] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:49:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:49:42.964783 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a72a9-7883-44d8-aead-fd427c9dc36e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:42.994662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 36/68] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:43.002210 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d98538-c03b-4ecd-8f5f-4f66b23dd3e0 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:43.100497 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:43.104823 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d98538-c03b-4ecd-8f5f-4f66b23dd3e0 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:43.115150 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a72a9-7883-44d8-aead-fd427c9dc36e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:43.150156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b334e0-9f67-4850-8870-19d20bb548be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:43.150973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b334e0-9f67-4850-8870-19d20bb548be None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:43.151721 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b334e0-9f67-4850-8870-19d20bb548be None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:43.151721 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b334e0-9f67-4850-8870-19d20bb548be None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:43.152466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b334e0-9f67-4850-8870-19d20bb548be None admin] RBAC: Adding query filter params (name=member) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:43.152880 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b334e0-9f67-4850-8870-19d20bb548be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:43.154423 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b334e0-9f67-4850-8870-19d20bb548be None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:43.160159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:43.169940 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 37/69] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:49:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:43.170584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605af84-4e20-4e71-b7e4-6e1d1dd7088a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:43.170996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b334e0-9f67-4850-8870-19d20bb548be None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:43.171879 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605af84-4e20-4e71-b7e4-6e1d1dd7088a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:43.172335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605af84-4e20-4e71-b7e4-6e1d1dd7088a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:43.172795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605af84-4e20-4e71-b7e4-6e1d1dd7088a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:43.185699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c605af84-4e20-4e71-b7e4-6e1d1dd7088a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:49:43.194995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c605af84-4e20-4e71-b7e4-6e1d1dd7088a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:49:43.194995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:49:43.194995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:49:43.194995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:49:43.194995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:49:43.194995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:49:43.194995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:49:43.194995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:49:43.194995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:49:43.194995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:49:43.194995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:49:43.194995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:49:43.194995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:49:43.194995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:49:43.194995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:49:43.194995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:49:43.195859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:49:43.195859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:49:43.195859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:49:43.195859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:49:43.195859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:49:43.195859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:49:43.195859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:49:43.195859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:49:43.195859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:49:43.195859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:49:43.195859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:49:43.195859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:49:43.195859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:49:43.195859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:49:43.195859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:49:43.195859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:49:43.196620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:49:43.196620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:49:43.196620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:49:43.196620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:49:43.196620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:49:43.196620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:49:43.196620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:49:43.196620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:49:43.196620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:49:43.196620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:49:43.196620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:49:43.196620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:49:43.199145 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 33/70] 10.222.0.76 () {56 vars in 1114 bytes} [Thu May 26 17:49:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:49:43.200171 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605af84-4e20-4e71-b7e4-6e1d1dd7088a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:43.281055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b334e0-9f67-4850-8870-19d20bb548be None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:43.316503 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605af84-4e20-4e71-b7e4-6e1d1dd7088a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:43.318258 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea4f60e-7140-4b05-9a47-d4f0f18dae9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:43.319079 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4f60e-7140-4b05-9a47-d4f0f18dae9f None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:43.319370 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4f60e-7140-4b05-9a47-d4f0f18dae9f None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:43.319648 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4f60e-7140-4b05-9a47-d4f0f18dae9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:43.332519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea4f60e-7140-4b05-9a47-d4f0f18dae9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:49:43.342387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-0ea4f60e-7140-4b05-9a47-d4f0f18dae9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:49:43.342387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:49:43.342387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:49:43.342387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:49:43.342387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:49:43.342387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:49:43.342387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:49:43.342387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:49:43.342387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:49:43.342387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:49:43.342387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:49:43.342387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:49:43.342387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:49:43.342387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:49:43.342387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:49:43.342387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:49:43.343509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:49:43.343509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:49:43.343509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:49:43.343509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:49:43.343509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:49:43.343509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:49:43.343509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:49:43.343509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:49:43.343509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:49:43.343509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:49:43.343509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:49:43.343509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:49:43.343509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:49:43.343509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:49:43.343509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:49:43.343509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:49:43.344312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:49:43.344312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:49:43.344312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:49:43.344312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:49:43.344312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:49:43.344312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:49:43.344312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:49:43.344312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:49:43.344312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:49:43.344312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:49:43.344312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:49:43.344312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:49:43.344312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 38/71] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:49:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:49:43.344949 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea4f60e-7140-4b05-9a47-d4f0f18dae9f None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:43.359316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381dd659-9b99-4b73-b893-78d547ff6c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:43.360747 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381dd659-9b99-4b73-b893-78d547ff6c23 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:43.361215 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381dd659-9b99-4b73-b893-78d547ff6c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:43.361689 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381dd659-9b99-4b73-b893-78d547ff6c23 None admin] PATH_INFO: `/v3/users/95b4e2fbe1aa4fa5b7cebed5b01639cd` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:43.386047 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381dd659-9b99-4b73-b893-78d547ff6c23 None admin] RBAC: Authorizing `identity:get_user(user_id=95b4e2fbe1aa4fa5b7cebed5b01639cd)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:43.388708 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381dd659-9b99-4b73-b893-78d547ff6c23 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:43.392048 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 34/72] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:49:43 2022] GET /identity/v3/users/95b4e2fbe1aa4fa5b7cebed5b01639cd => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:43.394361 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad328840-c82e-4069-bae9-964e8f5ac212 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:43.394764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad328840-c82e-4069-bae9-964e8f5ac212 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:43.395236 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad328840-c82e-4069-bae9-964e8f5ac212 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:43.398491 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 35/73] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:43.400473 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b3f6d-0b20-4623-8ca6-e842529a086e None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:43.400939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b3f6d-0b20-4623-8ca6-e842529a086e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:43.401383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b3f6d-0b20-4623-8ca6-e842529a086e None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:43.403868 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 36/74] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:43.405154 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b3f6d-0b20-4623-8ca6-e842529a086e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:43.418537 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:43.435961 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:43.481786 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea4f60e-7140-4b05-9a47-d4f0f18dae9f None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:43.511940 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b3f6d-0b20-4623-8ca6-e842529a086e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:43.516702 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f107bcb-8499-42d3-b3c8-bb837138e1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:43.517384 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f107bcb-8499-42d3-b3c8-bb837138e1f8 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:43.517674 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f107bcb-8499-42d3-b3c8-bb837138e1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:43.517960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f107bcb-8499-42d3-b3c8-bb837138e1f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:43.518755 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f107bcb-8499-42d3-b3c8-bb837138e1f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:43.519074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f107bcb-8499-42d3-b3c8-bb837138e1f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:43.520583 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f107bcb-8499-42d3-b3c8-bb837138e1f8 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:43.529043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 39/75] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:49:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:43.530066 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f107bcb-8499-42d3-b3c8-bb837138e1f8 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:43.552359 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9fc61-8452-40d8-8482-2d6f341cc121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:43.553043 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9fc61-8452-40d8-8482-2d6f341cc121 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:43.553331 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9fc61-8452-40d8-8482-2d6f341cc121 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:43.553618 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9fc61-8452-40d8-8482-2d6f341cc121 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:43.554405 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9fc61-8452-40d8-8482-2d6f341cc121 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:43.554719 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9fc61-8452-40d8-8482-2d6f341cc121 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:43.556122 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9fc61-8452-40d8-8482-2d6f341cc121 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:43.564352 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 37/76] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:49:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:43.564352 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aeb367-3545-4937-9dbd-3da88d9934fd None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:43.564352 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aeb367-3545-4937-9dbd-3da88d9934fd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:43.564352 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aeb367-3545-4937-9dbd-3da88d9934fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:43.582276 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e3aeb367-3545-4937-9dbd-3da88d9934fd None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:43.584026 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e3aeb367-3545-4937-9dbd-3da88d9934fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:43.630719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f107bcb-8499-42d3-b3c8-bb837138e1f8 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:43.648145 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:43.652905 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 38/77] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:43.654170 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2354432-7f9b-4c0b-a18e-7a5ce1e1a9c6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:43.654497 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2354432-7f9b-4c0b-a18e-7a5ce1e1a9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:43.654773 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2354432-7f9b-4c0b-a18e-7a5ce1e1a9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:43.654976 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:43.668434 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760d65f-a655-4d2e-a48d-2b4b1b8d68df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:43.669097 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760d65f-a655-4d2e-a48d-2b4b1b8d68df None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:43.669373 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760d65f-a655-4d2e-a48d-2b4b1b8d68df None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:43.669655 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760d65f-a655-4d2e-a48d-2b4b1b8d68df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:43.679148 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f2354432-7f9b-4c0b-a18e-7a5ce1e1a9c6 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:43.680337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f2354432-7f9b-4c0b-a18e-7a5ce1e1a9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:43.740071 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760d65f-a655-4d2e-a48d-2b4b1b8d68df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:43.741604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760d65f-a655-4d2e-a48d-2b4b1b8d68df None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:43.749836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 39/78] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:43.750785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2354432-7f9b-4c0b-a18e-7a5ce1e1a9c6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:43.859859 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 40/79] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:49:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:49:43.860750 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760d65f-a655-4d2e-a48d-2b4b1b8d68df None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:43.866439 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2354432-7f9b-4c0b-a18e-7a5ce1e1a9c6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:43.887933 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:43.902222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:43.909053 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6cfa6-0dd4-4e74-9f9d-14ebe6185fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:43.910245 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6cfa6-0dd4-4e74-9f9d-14ebe6185fcd None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:43.910541 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6cfa6-0dd4-4e74-9f9d-14ebe6185fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:43.910822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6cfa6-0dd4-4e74-9f9d-14ebe6185fcd None admin] PATH_INFO: `/v3/users/95b4e2fbe1aa4fa5b7cebed5b01639cd` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:43.913030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6cfa6-0dd4-4e74-9f9d-14ebe6185fcd None admin] RBAC: Authorizing `identity:get_user(user_id=95b4e2fbe1aa4fa5b7cebed5b01639cd)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:43.914639 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6cfa6-0dd4-4e74-9f9d-14ebe6185fcd None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:43.916361 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 40/80] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:49:43 2022] GET /identity/v3/users/95b4e2fbe1aa4fa5b7cebed5b01639cd => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:43.917171 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6cfa6-0dd4-4e74-9f9d-14ebe6185fcd None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:43.979691 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760d65f-a655-4d2e-a48d-2b4b1b8d68df None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:44.016035 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1879314-b426-4737-8d09-802d14dcc312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:44.016939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1879314-b426-4737-8d09-802d14dcc312 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:44.017229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1879314-b426-4737-8d09-802d14dcc312 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:44.017507 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1879314-b426-4737-8d09-802d14dcc312 None admin] PATH_INFO: `/v3/users/3c37098f90fe47ab88491e90626539d9` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:44.019096 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1879314-b426-4737-8d09-802d14dcc312 None admin] RBAC: Authorizing `identity:get_user(user_id=3c37098f90fe47ab88491e90626539d9)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:44.021316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1879314-b426-4737-8d09-802d14dcc312 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:44.023107 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 41/81] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:49:43 2022] GET /identity/v3/users/3c37098f90fe47ab88491e90626539d9 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:44.027857 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1879314-b426-4737-8d09-802d14dcc312 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:44.030309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6cfa6-0dd4-4e74-9f9d-14ebe6185fcd None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:44.068025 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f137d-f6cd-49c3-bee0-d3360954cfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:44.068940 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f137d-f6cd-49c3-bee0-d3360954cfd6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:44.069234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f137d-f6cd-49c3-bee0-d3360954cfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:44.069513 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f137d-f6cd-49c3-bee0-d3360954cfd6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:44.070828 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f137d-f6cd-49c3-bee0-d3360954cfd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:44.072331 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f137d-f6cd-49c3-bee0-d3360954cfd6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:44.074295 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 41/82] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:49:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:44.074918 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f137d-f6cd-49c3-bee0-d3360954cfd6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:44.133284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1879314-b426-4737-8d09-802d14dcc312 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:44.135021 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:44.151990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:44.170377 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb840e29-e487-4fa5-a020-601a3e991f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:44.171107 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb840e29-e487-4fa5-a020-601a3e991f9f None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:44.171391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb840e29-e487-4fa5-a020-601a3e991f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:44.171668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb840e29-e487-4fa5-a020-601a3e991f9f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:44.182232 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb840e29-e487-4fa5-a020-601a3e991f9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:49:44.189029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f137d-f6cd-49c3-bee0-d3360954cfd6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:44.189835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-eb840e29-e487-4fa5-a020-601a3e991f9f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:49:44.189835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:49:44.189835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:49:44.189835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:49:44.189835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:49:44.189835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:49:44.189835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:49:44.189835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:49:44.189835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:49:44.189835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:49:44.189835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:49:44.189835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:49:44.189835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:49:44.189835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:49:44.189835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:49:44.189835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:49:44.190887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:49:44.190887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:49:44.190887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:49:44.190887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:49:44.190887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:49:44.190887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:49:44.190887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:49:44.190887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:49:44.190887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:49:44.190887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:49:44.190887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:49:44.190887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:49:44.190887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:49:44.190887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:49:44.190887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:49:44.190887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:49:44.191753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:49:44.191753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:49:44.191753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:49:44.191753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:49:44.191753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:49:44.191753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:49:44.191753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:49:44.191753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:49:44.191753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:49:44.191753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:49:44.191753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:49:44.191753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:49:44.193252 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 42/83] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:49:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:49:44.194398 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb840e29-e487-4fa5-a020-601a3e991f9f None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:44.225669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d2818-15fa-4890-98a9-d4f605292947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:44.226568 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d2818-15fa-4890-98a9-d4f605292947 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:44.226857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d2818-15fa-4890-98a9-d4f605292947 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:44.227139 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d2818-15fa-4890-98a9-d4f605292947 None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:44.228339 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d2818-15fa-4890-98a9-d4f605292947 None admin] RBAC: Authorizing `identity:get_project(project_id=89f44d18dc9b464faeb1bf0bd03569af)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:44.229813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d2818-15fa-4890-98a9-d4f605292947 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:44.231565 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 42/84] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:49:44 2022] GET /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:44.232389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d2818-15fa-4890-98a9-d4f605292947 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:44.302512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb840e29-e487-4fa5-a020-601a3e991f9f None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:44.336234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69b7ab-a8e9-440e-92b4-f7fc9287caab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:44.336883 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69b7ab-a8e9-440e-92b4-f7fc9287caab None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:44.337173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69b7ab-a8e9-440e-92b4-f7fc9287caab None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:44.337456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69b7ab-a8e9-440e-92b4-f7fc9287caab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:44.337798 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d2818-15fa-4890-98a9-d4f605292947 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:44.356500 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:44.372561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9e195-592a-4c4c-af18-6dad637df21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:44.373224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9e195-592a-4c4c-af18-6dad637df21a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:44.373516 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9e195-592a-4c4c-af18-6dad637df21a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:44.373806 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9e195-592a-4c4c-af18-6dad637df21a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:44.376583 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:44.407747 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69b7ab-a8e9-440e-92b4-f7fc9287caab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:44.409238 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69b7ab-a8e9-440e-92b4-f7fc9287caab None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:44.445121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d9e195-592a-4c4c-af18-6dad637df21a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:44.446541 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d9e195-592a-4c4c-af18-6dad637df21a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:44.507719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 43/85] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:49:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:49:44.508569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69b7ab-a8e9-440e-92b4-f7fc9287caab None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:44.549464 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 43/86] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:49:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:49:44.554755 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9e195-592a-4c4c-af18-6dad637df21a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:44.558825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:44.587001 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:44.609561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69b7ab-a8e9-440e-92b4-f7fc9287caab None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:44.644464 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb36a6-f4b1-4dbe-ab40-668fd979fa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:44.645148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb36a6-f4b1-4dbe-ab40-668fd979fa0f None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:44.645966 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb36a6-f4b1-4dbe-ab40-668fd979fa0f None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:44.646269 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb36a6-f4b1-4dbe-ab40-668fd979fa0f None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:44.646997 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb36a6-f4b1-4dbe-ab40-668fd979fa0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:44.648475 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb36a6-f4b1-4dbe-ab40-668fd979fa0f None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:44.653565 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9e195-592a-4c4c-af18-6dad637df21a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:44.669680 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 44/87] 10.222.0.76 () {60 vars in 1132 bytes} [Thu May 26 17:49:44 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:49:44.670547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb36a6-f4b1-4dbe-ab40-668fd979fa0f None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:44.693948 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41c086-c455-431d-91eb-d384ada13a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:44.694758 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41c086-c455-431d-91eb-d384ada13a0e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:44.695046 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41c086-c455-431d-91eb-d384ada13a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:44.695427 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41c086-c455-431d-91eb-d384ada13a0e None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:44.696214 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae41c086-c455-431d-91eb-d384ada13a0e None admin] RBAC: Adding query filter params (name=member) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:44.696475 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae41c086-c455-431d-91eb-d384ada13a0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:44.697953 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae41c086-c455-431d-91eb-d384ada13a0e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:44.708618 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 44/88] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:49:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:44.709515 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41c086-c455-431d-91eb-d384ada13a0e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:44.806814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb36a6-f4b1-4dbe-ab40-668fd979fa0f None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:44.815567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:44.819445 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:44.827819 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41c086-c455-431d-91eb-d384ada13a0e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:44.849381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1b868-ca24-43aa-8ec9-83445d680519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:44.850163 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1b868-ca24-43aa-8ec9-83445d680519 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:44.850483 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1b868-ca24-43aa-8ec9-83445d680519 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:44.850764 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1b868-ca24-43aa-8ec9-83445d680519 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:44.851614 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f1b868-ca24-43aa-8ec9-83445d680519 None admin] RBAC: Adding query filter params (role.id=eb1858a5621543028836223abcb3c0a1, scope.project.id=89f44d18dc9b464faeb1bf0bd03569af, user.id=95b4e2fbe1aa4fa5b7cebed5b01639cd) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:44.852282 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f1b868-ca24-43aa-8ec9-83445d680519 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:44.853794 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f1b868-ca24-43aa-8ec9-83445d680519 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:44.863357 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 45/89] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:49:44 2022] GET /identity/v3/role_assignments?user.id=95b4e2fbe1aa4fa5b7cebed5b01639cd&scope.project.id=89f44d18dc9b464faeb1bf0bd03569af&role.id=eb1858a5621543028836223abcb3c0a1 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:44.864880 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1b868-ca24-43aa-8ec9-83445d680519 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:44.867873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98639349-f733-4ff3-b65f-0fa06a3fd6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:44.868771 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98639349-f733-4ff3-b65f-0fa06a3fd6cd None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:44.869062 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98639349-f733-4ff3-b65f-0fa06a3fd6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:44.869348 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98639349-f733-4ff3-b65f-0fa06a3fd6cd None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:44.870732 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98639349-f733-4ff3-b65f-0fa06a3fd6cd None admin] RBAC: Authorizing `identity:get_project(project_id=89f44d18dc9b464faeb1bf0bd03569af)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:44.872251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98639349-f733-4ff3-b65f-0fa06a3fd6cd None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:44.874100 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 45/90] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:49:44 2022] GET /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:44.875975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98639349-f733-4ff3-b65f-0fa06a3fd6cd None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:45.001862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1b868-ca24-43aa-8ec9-83445d680519 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:45.014390 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98639349-f733-4ff3-b65f-0fa06a3fd6cd None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:45.051116 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b0e0d-0aa0-4bca-932b-66fcab167d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:45.052864 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b0e0d-0aa0-4bca-932b-66fcab167d56 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:45.053628 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b0e0d-0aa0-4bca-932b-66fcab167d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:45.054070 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b0e0d-0aa0-4bca-932b-66fcab167d56 None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:45.055677 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b0e0d-0aa0-4bca-932b-66fcab167d56 None admin] RBAC: Authorizing `identity:get_project(project_id=89f44d18dc9b464faeb1bf0bd03569af)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:45.057831 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b0e0d-0aa0-4bca-932b-66fcab167d56 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:45.059892 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 46/91] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:49:44 2022] GET /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:45.060821 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b0e0d-0aa0-4bca-932b-66fcab167d56 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:45.062810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb8805-d6ca-4a67-b904-3546a3ad3866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:45.063781 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb8805-d6ca-4a67-b904-3546a3ad3866 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:45.064100 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb8805-d6ca-4a67-b904-3546a3ad3866 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:45.064410 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb8805-d6ca-4a67-b904-3546a3ad3866 None admin] PATH_INFO: `/v3/users/95b4e2fbe1aa4fa5b7cebed5b01639cd` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:45.065766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb8805-d6ca-4a67-b904-3546a3ad3866 None admin] RBAC: Authorizing `identity:get_user(user_id=95b4e2fbe1aa4fa5b7cebed5b01639cd)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:45.067924 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb8805-d6ca-4a67-b904-3546a3ad3866 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:45.070288 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 46/92] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:49:44 2022] GET /identity/v3/users/95b4e2fbe1aa4fa5b7cebed5b01639cd => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:45.073112 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb8805-d6ca-4a67-b904-3546a3ad3866 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:45.117699 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:45.122784 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:45.203368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb8805-d6ca-4a67-b904-3546a3ad3866 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:45.217471 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b0e0d-0aa0-4bca-932b-66fcab167d56 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:45.243812 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330c5e80-6aa5-4510-861b-9d6b82ec1fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:45.244483 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c5e80-6aa5-4510-861b-9d6b82ec1fde None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:45.244780 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c5e80-6aa5-4510-861b-9d6b82ec1fde None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:45.245063 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c5e80-6aa5-4510-861b-9d6b82ec1fde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:45.245909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330c5e80-6aa5-4510-861b-9d6b82ec1fde None admin] RBAC: Adding query filter params (role.id=4fa8f06085c741a5b3cf64e63cc0754f, scope.project.id=89f44d18dc9b464faeb1bf0bd03569af, user.id=3c37098f90fe47ab88491e90626539d9) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:45.246256 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330c5e80-6aa5-4510-861b-9d6b82ec1fde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:45.247749 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330c5e80-6aa5-4510-861b-9d6b82ec1fde None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:45.255512 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 47/93] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:49:45 2022] GET /identity/v3/role_assignments?user.id=3c37098f90fe47ab88491e90626539d9&scope.project.id=89f44d18dc9b464faeb1bf0bd03569af&role.id=4fa8f06085c741a5b3cf64e63cc0754f => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:45.256364 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330c5e80-6aa5-4510-861b-9d6b82ec1fde None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:45.262680 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0a5b7-b3a4-4d84-8791-5b3dfb4a224f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:45.263369 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0a5b7-b3a4-4d84-8791-5b3dfb4a224f None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:45.263659 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0a5b7-b3a4-4d84-8791-5b3dfb4a224f None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:45.263937 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0a5b7-b3a4-4d84-8791-5b3dfb4a224f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:45.264864 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0a5b7-b3a4-4d84-8791-5b3dfb4a224f None admin] RBAC: Adding query filter params (role.id=e731efeb1d3c407fb58cf64a02d95796, scope.project.id=89f44d18dc9b464faeb1bf0bd03569af, user.id=95b4e2fbe1aa4fa5b7cebed5b01639cd) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:45.265187 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0a5b7-b3a4-4d84-8791-5b3dfb4a224f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:45.266768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0a5b7-b3a4-4d84-8791-5b3dfb4a224f None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:45.276996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 47/94] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:49:45 2022] GET /identity/v3/role_assignments?user.id=95b4e2fbe1aa4fa5b7cebed5b01639cd&scope.project.id=89f44d18dc9b464faeb1bf0bd03569af&role.id=e731efeb1d3c407fb58cf64a02d95796 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:45.442889 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:45.450476 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330c5e80-6aa5-4510-861b-9d6b82ec1fde None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:45.500556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facd8ec-3516-4479-90b3-fed00a3bf8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:45.501246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facd8ec-3516-4479-90b3-fed00a3bf8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:45.501536 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facd8ec-3516-4479-90b3-fed00a3bf8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:45.501819 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facd8ec-3516-4479-90b3-fed00a3bf8c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:45.588875 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8facd8ec-3516-4479-90b3-fed00a3bf8c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:45.590308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8facd8ec-3516-4479-90b3-fed00a3bf8c0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:45.704405 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:45.712587 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 48/95] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:49:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:49:45.718243 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0a5b7-b3a4-4d84-8791-5b3dfb4a224f None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:45.891128 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0a5b7-b3a4-4d84-8791-5b3dfb4a224f None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:45.892686 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:45.939215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d947b3-3582-40d9-8482-4070ee1cccc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:45.940097 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d947b3-3582-40d9-8482-4070ee1cccc2 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:45.940389 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d947b3-3582-40d9-8482-4070ee1cccc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:45.940674 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d947b3-3582-40d9-8482-4070ee1cccc2 None admin] PATH_INFO: `/v3/projects/94a321d75caa43cc94d0231c3442718f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:45.947123 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d947b3-3582-40d9-8482-4070ee1cccc2 None admin] RBAC: Authorizing `identity:get_project(project_id=94a321d75caa43cc94d0231c3442718f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:45.948631 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d947b3-3582-40d9-8482-4070ee1cccc2 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:45.950473 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 48/96] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:49:45 2022] GET /identity/v3/projects/94a321d75caa43cc94d0231c3442718f => generated 312 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:45.956428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facd8ec-3516-4479-90b3-fed00a3bf8c0 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:46.121932 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facd8ec-3516-4479-90b3-fed00a3bf8c0 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:46.174992 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a26b71-ca5a-4bb3-a1a7-e0bd9b0b2497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:46.175664 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a26b71-ca5a-4bb3-a1a7-e0bd9b0b2497 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:46.175955 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a26b71-ca5a-4bb3-a1a7-e0bd9b0b2497 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:46.176238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a26b71-ca5a-4bb3-a1a7-e0bd9b0b2497 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:46.177620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a26b71-ca5a-4bb3-a1a7-e0bd9b0b2497 None admin] RBAC: Adding query filter params (role.id=eb1858a5621543028836223abcb3c0a1, scope.project.id=94a321d75caa43cc94d0231c3442718f, user.id=95b4e2fbe1aa4fa5b7cebed5b01639cd) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:46.177938 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a26b71-ca5a-4bb3-a1a7-e0bd9b0b2497 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:46.179421 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a26b71-ca5a-4bb3-a1a7-e0bd9b0b2497 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:46.187445 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 49/97] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:49:45 2022] GET /identity/v3/role_assignments?user.id=95b4e2fbe1aa4fa5b7cebed5b01639cd&scope.project.id=94a321d75caa43cc94d0231c3442718f&role.id=eb1858a5621543028836223abcb3c0a1 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:47.999838 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86129dbe-e7f8-44e5-85f5-ef7a53fe8563 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:48.000371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86129dbe-e7f8-44e5-85f5-ef7a53fe8563 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:48.000469 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86129dbe-e7f8-44e5-85f5-ef7a53fe8563 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:48.002018 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 49/98] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:48.008413 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcece56-e4c7-420c-b514-cbe367fcbbcb None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:48.008908 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcece56-e4c7-420c-b514-cbe367fcbbcb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:48.008998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcece56-e4c7-420c-b514-cbe367fcbbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:48.031443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fb693-53c5-4224-98bd-588bba96ee03 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:48.031753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fb693-53c5-4224-98bd-588bba96ee03 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:48.032028 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fb693-53c5-4224-98bd-588bba96ee03 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:48.033565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 50/99] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:49:48.034769 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4fcece56-e4c7-420c-b514-cbe367fcbbcb None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:48.037007 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4fcece56-e4c7-420c-b514-cbe367fcbbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:48.039954 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3e406-e7d5-4c23-a4b4-28da575f69b7 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:48.040251 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3e406-e7d5-4c23-a4b4-28da575f69b7 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:48.040541 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3e406-e7d5-4c23-a4b4-28da575f69b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:48.057898 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-86c3e406-e7d5-4c23-a4b4-28da575f69b7 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:48.059142 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-86c3e406-e7d5-4c23-a4b4-28da575f69b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:48.176577 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:48.210582 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 51/100] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:49:48.212087 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 50/101] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:48.236276 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d01099-1559-4534-a6af-fb08207cf1e1 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:48.236570 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d01099-1559-4534-a6af-fb08207cf1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:48.236856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d01099-1559-4534-a6af-fb08207cf1e1 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:48.238400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 52/102] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:48.248492 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d01099-1559-4534-a6af-fb08207cf1e1 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:48.250076 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9e3e9-b4d2-4186-a924-01cae216f5c7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:48.250569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9e3e9-b4d2-4186-a924-01cae216f5c7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:48.251192 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9e3e9-b4d2-4186-a924-01cae216f5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:48.287724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-89f9e3e9-b4d2-4186-a924-01cae216f5c7 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:48.288955 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-89f9e3e9-b4d2-4186-a924-01cae216f5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:48.411248 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:48.419259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d01099-1559-4534-a6af-fb08207cf1e1 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:48.423567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 51/103] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:48.432418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9e3e9-b4d2-4186-a924-01cae216f5c7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:48.497198 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f2e6f-738f-47e0-aa81-0eae0625bed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:48.498457 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f2e6f-738f-47e0-aa81-0eae0625bed4 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:48.500180 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f2e6f-738f-47e0-aa81-0eae0625bed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:48.501366 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f2e6f-738f-47e0-aa81-0eae0625bed4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:48.505365 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f2e6f-738f-47e0-aa81-0eae0625bed4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:48.507794 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f2e6f-738f-47e0-aa81-0eae0625bed4 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:48.510344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 53/104] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:49:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:48.517644 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f2e6f-738f-47e0-aa81-0eae0625bed4 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:48.636665 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9e3e9-b4d2-4186-a924-01cae216f5c7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:48.642171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f2e6f-738f-47e0-aa81-0eae0625bed4 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:48.664184 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:48.682176 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcebd5-bde3-4534-a529-9591164f5a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:48.683300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcebd5-bde3-4534-a529-9591164f5a53 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:48.683743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcebd5-bde3-4534-a529-9591164f5a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:48.684172 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcebd5-bde3-4534-a529-9591164f5a53 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:48.696309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bc0a8-b123-46a3-a730-a27632ca08d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:48.697140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bc0a8-b123-46a3-a730-a27632ca08d5 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:48.697430 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bc0a8-b123-46a3-a730-a27632ca08d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:48.697713 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bc0a8-b123-46a3-a730-a27632ca08d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:48.699237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcebd5-bde3-4534-a529-9591164f5a53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:49:48.709575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-d9bcebd5-bde3-4534-a529-9591164f5a53 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:49:48.709575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:49:48.709575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:49:48.709575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:49:48.709575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:49:48.709575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:49:48.709575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:49:48.709575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:49:48.709575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:49:48.709575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:49:48.709575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:49:48.709575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:49:48.709575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:49:48.709575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:49:48.709575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:49:48.709575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:49:48.710745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:49:48.710745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:49:48.710745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:49:48.710745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:49:48.710745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:49:48.710745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:49:48.710745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:49:48.710745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:49:48.710745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:49:48.710745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:49:48.710745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:49:48.710745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:49:48.710745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:49:48.710745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:49:48.710745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:49:48.710745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:49:48.711528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:49:48.711528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:49:48.711528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:49:48.711528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:49:48.711528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:49:48.711528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:49:48.711528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:49:48.711528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:49:48.711528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:49:48.711528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:49:48.711528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:49:48.711528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:49:48.712155 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bc0a8-b123-46a3-a730-a27632ca08d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:49:48.714044 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 52/105] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:49:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:49:48.715411 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcebd5-bde3-4534-a529-9591164f5a53 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:48.721396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-886bc0a8-b123-46a3-a730-a27632ca08d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:49:48.721396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:49:48.721396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:49:48.721396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:49:48.721396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:49:48.721396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:49:48.721396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:49:48.721396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:49:48.721396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:49:48.721396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:49:48.721396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:49:48.721396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:49:48.721396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:49:48.721396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:49:48.721396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:49:48.721396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:49:48.722429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:49:48.722429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:49:48.722429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:49:48.722429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:49:48.722429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:49:48.722429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:49:48.722429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:49:48.722429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:49:48.722429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:49:48.722429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:49:48.722429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:49:48.722429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:49:48.722429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:49:48.722429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:49:48.722429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:49:48.722429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:49:48.727559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:49:48.727559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:49:48.727559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:49:48.727559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:49:48.727559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:49:48.727559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:49:48.727559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:49:48.727559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:49:48.727559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:49:48.727559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:49:48.727559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:49:48.727559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:49:48.727559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 54/106] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:49:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:49:48.727559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e1aaa-4535-467e-8358-da5edcd2f5ea None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:48.728693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e1aaa-4535-467e-8358-da5edcd2f5ea None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:48.728693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e1aaa-4535-467e-8358-da5edcd2f5ea None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:48.728693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 55/107] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:48.728693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e1aaa-4535-467e-8358-da5edcd2f5ea None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:48.778258 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:48.863360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcebd5-bde3-4534-a529-9591164f5a53 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:48.872845 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e1aaa-4535-467e-8358-da5edcd2f5ea None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:48.913663 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e939d-6219-41eb-81f5-c6183a9134e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:48.914476 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e939d-6219-41eb-81f5-c6183a9134e4 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:48.914777 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e939d-6219-41eb-81f5-c6183a9134e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:48.915064 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e939d-6219-41eb-81f5-c6183a9134e4 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:48.915814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e939d-6219-41eb-81f5-c6183a9134e4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:48.917287 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386c2ad1-9bc4-440e-ab05-3d586765878d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:48.917523 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e939d-6219-41eb-81f5-c6183a9134e4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:48.918430 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c2ad1-9bc4-440e-ab05-3d586765878d None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:48.919038 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c2ad1-9bc4-440e-ab05-3d586765878d None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:48.919673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c2ad1-9bc4-440e-ab05-3d586765878d None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:48.920661 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386c2ad1-9bc4-440e-ab05-3d586765878d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:48.921276 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386c2ad1-9bc4-440e-ab05-3d586765878d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:48.923164 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386c2ad1-9bc4-440e-ab05-3d586765878d None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:48.937606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 56/108] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:49:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:48.938863 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa358f-e2b9-4946-b163-82140998ac02 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:48.939159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa358f-e2b9-4946-b163-82140998ac02 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:48.939443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa358f-e2b9-4946-b163-82140998ac02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:48.951040 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 53/109] 10.222.0.76 () {60 vars in 1141 bytes} [Thu May 26 17:49:48 2022] POST /identity/v3/projects => generated 302 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:49:48.958456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-8cfa358f-e2b9-4946-b163-82140998ac02 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:48.959471 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e939d-6219-41eb-81f5-c6183a9134e4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:48.959634 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-8cfa358f-e2b9-4946-b163-82140998ac02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:49.012332 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:49.061785 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 57/110] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:49.062853 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa358f-e2b9-4946-b163-82140998ac02 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:49.089498 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:49.129227 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e939d-6219-41eb-81f5-c6183a9134e4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:49.180574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91e7ad-7ea0-4549-9a8d-8d01fbb1482c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:49.181260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91e7ad-7ea0-4549-9a8d-8d01fbb1482c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:49.181551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91e7ad-7ea0-4549-9a8d-8d01fbb1482c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:49.181877 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91e7ad-7ea0-4549-9a8d-8d01fbb1482c None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:49.183071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91e7ad-7ea0-4549-9a8d-8d01fbb1482c None admin] RBAC: Adding query filter params (name=member) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:49.183071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91e7ad-7ea0-4549-9a8d-8d01fbb1482c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:49.185017 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91e7ad-7ea0-4549-9a8d-8d01fbb1482c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:49.194123 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 54/111] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:49:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:49.199975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa358f-e2b9-4946-b163-82140998ac02 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:49.200134 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54a789-1719-41ba-8f75-9d7be6476b50 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:49.200134 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54a789-1719-41ba-8f75-9d7be6476b50 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:49.200134 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54a789-1719-41ba-8f75-9d7be6476b50 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:49.200134 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 55/112] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:49.200134 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e54a789-1719-41ba-8f75-9d7be6476b50 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:49.243636 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9dfa7d-736e-4dfd-a74e-ec2a8eefa6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:49.245015 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9dfa7d-736e-4dfd-a74e-ec2a8eefa6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:49.245478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9dfa7d-736e-4dfd-a74e-ec2a8eefa6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:49.245932 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9dfa7d-736e-4dfd-a74e-ec2a8eefa6b0 None admin] PATH_INFO: `/v3/users/b087f9732f764f9a9f4c915abe5814c8` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:49.268479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9dfa7d-736e-4dfd-a74e-ec2a8eefa6b0 None admin] RBAC: Authorizing `identity:get_user(user_id=b087f9732f764f9a9f4c915abe5814c8)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:49.271375 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9dfa7d-736e-4dfd-a74e-ec2a8eefa6b0 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:49.274491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 58/113] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:49:49 2022] GET /identity/v3/users/b087f9732f764f9a9f4c915abe5814c8 => generated 289 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:49.275470 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9dfa7d-736e-4dfd-a74e-ec2a8eefa6b0 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:49.323816 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:49.333581 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e54a789-1719-41ba-8f75-9d7be6476b50 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:49.374027 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74551c13-b98d-4f6c-b68c-4d258ff2b2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:49.375284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74551c13-b98d-4f6c-b68c-4d258ff2b2bd None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:49.375805 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74551c13-b98d-4f6c-b68c-4d258ff2b2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:49.376326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74551c13-b98d-4f6c-b68c-4d258ff2b2bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:49.377030 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:49.389813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74551c13-b98d-4f6c-b68c-4d258ff2b2bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:49:49.399822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-74551c13-b98d-4f6c-b68c-4d258ff2b2bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:49:49.399822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:49:49.399822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:49:49.399822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:49:49.399822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:49:49.399822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:49:49.399822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:49:49.399822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:49:49.399822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:49:49.399822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:49:49.399822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:49:49.399822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:49:49.399822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:49:49.399822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:49:49.399822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:49:49.399822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:49:49.400793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:49:49.400793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:49:49.400793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:49:49.400793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:49:49.400793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:49:49.400793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:49:49.400793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:49:49.400793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:49:49.400793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:49:49.400793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:49:49.400793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:49:49.400793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:49:49.400793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:49:49.400793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:49:49.400793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:49:49.400793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:49:49.401550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:49:49.401550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:49:49.401550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:49:49.401550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:49:49.401550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:49:49.401550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:49:49.401550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:49:49.401550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:49:49.401550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:49:49.401550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:49:49.401550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:49:49.401550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:49:49.404176 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 56/114] 10.222.0.76 () {56 vars in 1114 bytes} [Thu May 26 17:49:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:49:49.404409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9dfa7d-736e-4dfd-a74e-ec2a8eefa6b0 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:49.405998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0056d-16bb-4b6b-a8aa-7a3e0ba65da4 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:49.406629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0056d-16bb-4b6b-a8aa-7a3e0ba65da4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:49.407110 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0056d-16bb-4b6b-a8aa-7a3e0ba65da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:49.429005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-62a0056d-16bb-4b6b-a8aa-7a3e0ba65da4 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:49.430678 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-62a0056d-16bb-4b6b-a8aa-7a3e0ba65da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:49.452594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41c869-f357-4096-a807-34e8cd4f6d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:49.453497 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41c869-f357-4096-a807-34e8cd4f6d5c None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:49.453789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41c869-f357-4096-a807-34e8cd4f6d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:49.454075 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41c869-f357-4096-a807-34e8cd4f6d5c None admin] PATH_INFO: `/v3/users/95b4e2fbe1aa4fa5b7cebed5b01639cd` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:49.455408 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e41c869-f357-4096-a807-34e8cd4f6d5c None admin] RBAC: Authorizing `identity:get_user(user_id=95b4e2fbe1aa4fa5b7cebed5b01639cd)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:49.456986 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e41c869-f357-4096-a807-34e8cd4f6d5c None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:49.464050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 59/115] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:49:49 2022] GET /identity/v3/users/95b4e2fbe1aa4fa5b7cebed5b01639cd => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:49.464050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41c869-f357-4096-a807-34e8cd4f6d5c None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:49.514215 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 57/116] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:49.515444 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0056d-16bb-4b6b-a8aa-7a3e0ba65da4 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:49.607859 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41c869-f357-4096-a807-34e8cd4f6d5c None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:49.608660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:49.626703 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0056d-16bb-4b6b-a8aa-7a3e0ba65da4 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:49.646419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593942a-e0a3-4a5e-9e13-8c94f1a965ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:49.647532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593942a-e0a3-4a5e-9e13-8c94f1a965ff None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:49.648034 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593942a-e0a3-4a5e-9e13-8c94f1a965ff None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:49.648390 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593942a-e0a3-4a5e-9e13-8c94f1a965ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:49.663115 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6f754-f9d3-4813-847e-969344f1a60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:49.663797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6f754-f9d3-4813-847e-969344f1a60b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:49.664087 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6f754-f9d3-4813-847e-969344f1a60b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:49.664367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6f754-f9d3-4813-847e-969344f1a60b None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:49.665126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb6f754-f9d3-4813-847e-969344f1a60b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:49.665437 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb6f754-f9d3-4813-847e-969344f1a60b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:49.666884 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb6f754-f9d3-4813-847e-969344f1a60b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:49.667936 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:49.675886 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 58/117] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:49:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:49.676751 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6f754-f9d3-4813-847e-969344f1a60b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:49.729444 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8593942a-e0a3-4a5e-9e13-8c94f1a965ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:49.730989 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8593942a-e0a3-4a5e-9e13-8c94f1a965ff None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:49.796579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6f754-f9d3-4813-847e-969344f1a60b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:49.834973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9416c04-b764-44d2-a5b0-4ca08e8c7c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:49.835985 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9416c04-b764-44d2-a5b0-4ca08e8c7c09 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:49.836277 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9416c04-b764-44d2-a5b0-4ca08e8c7c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:49.836560 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9416c04-b764-44d2-a5b0-4ca08e8c7c09 None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:49.837878 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9416c04-b764-44d2-a5b0-4ca08e8c7c09 None admin] RBAC: Authorizing `identity:get_project(project_id=89f44d18dc9b464faeb1bf0bd03569af)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:49.839284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9416c04-b764-44d2-a5b0-4ca08e8c7c09 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:49.841359 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 59/118] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:49:49 2022] GET /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:49.842216 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9416c04-b764-44d2-a5b0-4ca08e8c7c09 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:49.850223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 60/119] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:49:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:49:49.854525 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa491a-1f42-403f-bcb5-a6f4028be103 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:49.854824 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa491a-1f42-403f-bcb5-a6f4028be103 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:49.855114 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa491a-1f42-403f-bcb5-a6f4028be103 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:49.857695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 61/120] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:49.857695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa491a-1f42-403f-bcb5-a6f4028be103 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:49.861507 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:49.910773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:49.957408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9416c04-b764-44d2-a5b0-4ca08e8c7c09 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:49.974712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa491a-1f42-403f-bcb5-a6f4028be103 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:49.996517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb05d0d-655e-46d5-8304-268beab4a71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:49.997670 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb05d0d-655e-46d5-8304-268beab4a71f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:49.997670 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb05d0d-655e-46d5-8304-268beab4a71f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:49.997929 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb05d0d-655e-46d5-8304-268beab4a71f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:50.009273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb05d0d-655e-46d5-8304-268beab4a71f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:49:50.012364 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b47c91-e333-4514-90fd-7dd0cf3dd131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:50.013303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b47c91-e333-4514-90fd-7dd0cf3dd131 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:50.013657 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b47c91-e333-4514-90fd-7dd0cf3dd131 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:50.014004 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b47c91-e333-4514-90fd-7dd0cf3dd131 None admin] PATH_INFO: `/v3/users/95b4e2fbe1aa4fa5b7cebed5b01639cd` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:50.015307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b47c91-e333-4514-90fd-7dd0cf3dd131 None admin] RBAC: Authorizing `identity:get_user(user_id=95b4e2fbe1aa4fa5b7cebed5b01639cd)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:50.016868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b47c91-e333-4514-90fd-7dd0cf3dd131 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:50.017282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-7fb05d0d-655e-46d5-8304-268beab4a71f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:49:50.017282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:49:50.017282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:49:50.017282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:49:50.017282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:49:50.017282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:49:50.017282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:49:50.017282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:49:50.017282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:49:50.017282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:49:50.017282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:49:50.017282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:49:50.017282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:49:50.017282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:49:50.017282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:49:50.017282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:49:50.018557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:49:50.018557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:49:50.018557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:49:50.018557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:49:50.018557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:49:50.018557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:49:50.018557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:49:50.018557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:49:50.018557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:49:50.018557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:49:50.018557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:49:50.018557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:49:50.018557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:49:50.018557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:49:50.018557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:49:50.018557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:49:50.019337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:49:50.019337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:49:50.019337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:49:50.019337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:49:50.019337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:49:50.019337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:49:50.019337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:49:50.019337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:49:50.019337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:49:50.019337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:49:50.019337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:49:50.019337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:49:50.019917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 62/122] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:49:49 2022] GET /identity/v3/users/95b4e2fbe1aa4fa5b7cebed5b01639cd => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:50.020013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 60/122] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:49:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:49:50.020078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b47c91-e333-4514-90fd-7dd0cf3dd131 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:50.020148 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb05d0d-655e-46d5-8304-268beab4a71f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:50.112412 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:50.136358 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b47c91-e333-4514-90fd-7dd0cf3dd131 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:50.150037 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb05d0d-655e-46d5-8304-268beab4a71f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:50.151935 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:50.175058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c595c-e85b-40df-895d-dbce5aec08b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:50.175977 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c595c-e85b-40df-895d-dbce5aec08b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:50.176263 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c595c-e85b-40df-895d-dbce5aec08b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:50.176542 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c595c-e85b-40df-895d-dbce5aec08b3 None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af/users/95b4e2fbe1aa4fa5b7cebed5b01639cd/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:50.179810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c595c-e85b-40df-895d-dbce5aec08b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=89f44d18dc9b464faeb1bf0bd03569af, user_id=95b4e2fbe1aa4fa5b7cebed5b01639cd, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:50.181297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c595c-e85b-40df-895d-dbce5aec08b3 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:50.198294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 63/123] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:49:50 2022] PUT /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af/users/95b4e2fbe1aa4fa5b7cebed5b01639cd/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:49:50.205345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20291528-3268-49aa-898a-3dae57380102 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:50.205345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20291528-3268-49aa-898a-3dae57380102 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:50.205345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20291528-3268-49aa-898a-3dae57380102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:50.212114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b72d3b-d122-4873-b603-60fe5c48116a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:50.213020 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b72d3b-d122-4873-b603-60fe5c48116a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:50.213316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b72d3b-d122-4873-b603-60fe5c48116a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:50.213606 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b72d3b-d122-4873-b603-60fe5c48116a None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:50.214946 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b72d3b-d122-4873-b603-60fe5c48116a None admin] RBAC: Authorizing `identity:get_project(project_id=89f44d18dc9b464faeb1bf0bd03569af)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:50.216487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b72d3b-d122-4873-b603-60fe5c48116a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:50.218762 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 61/124] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:49:50 2022] GET /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:50.223428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b72d3b-d122-4873-b603-60fe5c48116a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:50.223776 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-20291528-3268-49aa-898a-3dae57380102 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:50.225186 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-20291528-3268-49aa-898a-3dae57380102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:50.330235 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 64/125] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:50.331227 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20291528-3268-49aa-898a-3dae57380102 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:50.350111 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b72d3b-d122-4873-b603-60fe5c48116a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:50.406576 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c7cde-7736-4b53-a304-d923c433973b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:50.407376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c7cde-7736-4b53-a304-d923c433973b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:50.407659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c7cde-7736-4b53-a304-d923c433973b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:50.407938 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c7cde-7736-4b53-a304-d923c433973b None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:50.409892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c7cde-7736-4b53-a304-d923c433973b None admin] RBAC: Authorizing `identity:get_project(project_id=89f44d18dc9b464faeb1bf0bd03569af)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:50.412119 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c7cde-7736-4b53-a304-d923c433973b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:50.415152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 62/126] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:49:50 2022] GET /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:50.416312 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c7cde-7736-4b53-a304-d923c433973b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:50.434067 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:50.478706 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20291528-3268-49aa-898a-3dae57380102 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:50.489428 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:50.530934 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086210a-fd54-4143-acaf-0d19ce1c85e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:50.532126 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086210a-fd54-4143-acaf-0d19ce1c85e1 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:50.532576 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086210a-fd54-4143-acaf-0d19ce1c85e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:50.533294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086210a-fd54-4143-acaf-0d19ce1c85e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:50.534645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0086210a-fd54-4143-acaf-0d19ce1c85e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:50.535361 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0086210a-fd54-4143-acaf-0d19ce1c85e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:50.539660 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0086210a-fd54-4143-acaf-0d19ce1c85e1 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:50.544949 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c7cde-7736-4b53-a304-d923c433973b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:50.551722 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 65/127] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:49:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:50.557204 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086210a-fd54-4143-acaf-0d19ce1c85e1 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:50.597442 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ccfbf-f9e8-4ef2-bdbc-8ab99a057b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:50.599121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ccfbf-f9e8-4ef2-bdbc-8ab99a057b10 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:50.599585 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ccfbf-f9e8-4ef2-bdbc-8ab99a057b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:50.600021 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ccfbf-f9e8-4ef2-bdbc-8ab99a057b10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:50.601045 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ccfbf-f9e8-4ef2-bdbc-8ab99a057b10 None admin] RBAC: Adding query filter params (role.id=5b3eae4902f44cc9a56bf2ec2a8e13f5, scope.project.id=89f44d18dc9b464faeb1bf0bd03569af, user.id=b087f9732f764f9a9f4c915abe5814c8) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:50.602238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ccfbf-f9e8-4ef2-bdbc-8ab99a057b10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:50.604691 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ccfbf-f9e8-4ef2-bdbc-8ab99a057b10 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:50.613390 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 63/128] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:49:50 2022] GET /identity/v3/role_assignments?user.id=b087f9732f764f9a9f4c915abe5814c8&scope.project.id=89f44d18dc9b464faeb1bf0bd03569af&role.id=5b3eae4902f44cc9a56bf2ec2a8e13f5 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:50.614558 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ccfbf-f9e8-4ef2-bdbc-8ab99a057b10 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:50.712862 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:50.745589 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ccfbf-f9e8-4ef2-bdbc-8ab99a057b10 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:50.773729 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086210a-fd54-4143-acaf-0d19ce1c85e1 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:50.794458 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289e370-618d-4647-a89f-c2f4210d6a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:50.795685 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289e370-618d-4647-a89f-c2f4210d6a3c None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:50.796125 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289e370-618d-4647-a89f-c2f4210d6a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:50.796567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289e370-618d-4647-a89f-c2f4210d6a3c None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af/users/95b4e2fbe1aa4fa5b7cebed5b01639cd/roles/e731efeb1d3c407fb58cf64a02d95796` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:50.799558 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289e370-618d-4647-a89f-c2f4210d6a3c None admin] RBAC: Authorizing `identity:create_grant(project_id=89f44d18dc9b464faeb1bf0bd03569af, user_id=95b4e2fbe1aa4fa5b7cebed5b01639cd, role_id=e731efeb1d3c407fb58cf64a02d95796)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:50.801743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289e370-618d-4647-a89f-c2f4210d6a3c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:50.826434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 64/129] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:49:50 2022] PUT /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af/users/95b4e2fbe1aa4fa5b7cebed5b01639cd/roles/e731efeb1d3c407fb58cf64a02d95796 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:49:50.828039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289e370-618d-4647-a89f-c2f4210d6a3c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:50.868595 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f01b6-05a5-456c-ba82-f5f63c5c9aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:50.869423 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f01b6-05a5-456c-ba82-f5f63c5c9aff None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:50.869716 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f01b6-05a5-456c-ba82-f5f63c5c9aff None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:50.869999 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f01b6-05a5-456c-ba82-f5f63c5c9aff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:50.907362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f01b6-05a5-456c-ba82-f5f63c5c9aff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:49:50.924371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-528f01b6-05a5-456c-ba82-f5f63c5c9aff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:49:50.924371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:49:50.924371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:49:50.924371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:49:50.924371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:49:50.924371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:49:50.924371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:49:50.924371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:49:50.924371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:49:50.924371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:49:50.924371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:49:50.924371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:49:50.924371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:49:50.924371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:49:50.924371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:49:50.924371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:49:50.925605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:49:50.925605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:49:50.925605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:49:50.925605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:49:50.925605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:49:50.925605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:49:50.925605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:49:50.925605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:49:50.925605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:49:50.925605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:49:50.925605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:49:50.925605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:49:50.925605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:49:50.925605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:49:50.925605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:49:50.925605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:49:50.926638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:49:50.926638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:49:50.926638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:49:50.926638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:49:50.926638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:49:50.926638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:49:50.926638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:49:50.926638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:49:50.926638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:49:50.926638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:49:50.926638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:49:50.926638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:49:50.926638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 66/130] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:49:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:49:50.935332 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f01b6-05a5-456c-ba82-f5f63c5c9aff None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:50.980175 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289e370-618d-4647-a89f-c2f4210d6a3c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:51.008094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:51.028053 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185554e-d910-4d4e-87fe-87d397f4c50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:51.029662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185554e-d910-4d4e-87fe-87d397f4c50d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:51.030148 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185554e-d910-4d4e-87fe-87d397f4c50d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:51.030926 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185554e-d910-4d4e-87fe-87d397f4c50d None admin] PATH_INFO: `/v3/users/3c37098f90fe47ab88491e90626539d9` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:51.032518 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2185554e-d910-4d4e-87fe-87d397f4c50d None admin] RBAC: Authorizing `identity:get_user(user_id=3c37098f90fe47ab88491e90626539d9)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:51.034494 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2185554e-d910-4d4e-87fe-87d397f4c50d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:51.037749 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 65/131] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:49:50 2022] GET /identity/v3/users/3c37098f90fe47ab88491e90626539d9 => generated 247 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:51.045051 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185554e-d910-4d4e-87fe-87d397f4c50d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:51.048558 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:51.172969 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f01b6-05a5-456c-ba82-f5f63c5c9aff None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:51.187773 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185554e-d910-4d4e-87fe-87d397f4c50d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:51.227970 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd83ba-6999-4f8f-825c-55ced2b43220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:51.230258 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd83ba-6999-4f8f-825c-55ced2b43220 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:51.231799 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd83ba-6999-4f8f-825c-55ced2b43220 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:51.232302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd83ba-6999-4f8f-825c-55ced2b43220 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:51.234906 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd83ba-6999-4f8f-825c-55ced2b43220 None admin] RBAC: Adding query filter params (name=member) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:51.235415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd83ba-6999-4f8f-825c-55ced2b43220 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:51.238301 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd83ba-6999-4f8f-825c-55ced2b43220 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:51.261188 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 67/132] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:49:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:51.267347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd83ba-6999-4f8f-825c-55ced2b43220 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:51.269067 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583e011-f437-4d44-91e7-cf26dd5c3a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:51.271163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583e011-f437-4d44-91e7-cf26dd5c3a78 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:51.273865 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583e011-f437-4d44-91e7-cf26dd5c3a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:51.274358 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583e011-f437-4d44-91e7-cf26dd5c3a78 None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:51.288261 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9583e011-f437-4d44-91e7-cf26dd5c3a78 None admin] RBAC: Authorizing `identity:get_project(project_id=89f44d18dc9b464faeb1bf0bd03569af)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:51.290272 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9583e011-f437-4d44-91e7-cf26dd5c3a78 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:51.292495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 66/133] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:49:51 2022] GET /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:49:51.306582 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583e011-f437-4d44-91e7-cf26dd5c3a78 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:51.452109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:51.467032 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd83ba-6999-4f8f-825c-55ced2b43220 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:51.523838 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c10d43-14de-4195-b8d3-44b997a7571c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:51.525125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c10d43-14de-4195-b8d3-44b997a7571c None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:51.525640 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c10d43-14de-4195-b8d3-44b997a7571c None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:51.526119 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c10d43-14de-4195-b8d3-44b997a7571c None admin] PATH_INFO: `/v3/users/95b4e2fbe1aa4fa5b7cebed5b01639cd` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:51.526239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:51.527712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c10d43-14de-4195-b8d3-44b997a7571c None admin] RBAC: Authorizing `identity:get_user(user_id=95b4e2fbe1aa4fa5b7cebed5b01639cd)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:51.534209 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c10d43-14de-4195-b8d3-44b997a7571c None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:51.537374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 68/134] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:49:51 2022] GET /identity/v3/users/95b4e2fbe1aa4fa5b7cebed5b01639cd => generated 275 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:51.551116 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c10d43-14de-4195-b8d3-44b997a7571c None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:51.580160 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583e011-f437-4d44-91e7-cf26dd5c3a78 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:51.638622 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26607e-2d51-4c7a-a72d-f2f1bad6ed1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:51.638622 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26607e-2d51-4c7a-a72d-f2f1bad6ed1c None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:51.642222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26607e-2d51-4c7a-a72d-f2f1bad6ed1c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:51.642222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26607e-2d51-4c7a-a72d-f2f1bad6ed1c None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af/users/3c37098f90fe47ab88491e90626539d9/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:51.642222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac26607e-2d51-4c7a-a72d-f2f1bad6ed1c None admin] RBAC: Authorizing `identity:create_grant(project_id=89f44d18dc9b464faeb1bf0bd03569af, user_id=3c37098f90fe47ab88491e90626539d9, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:51.648584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac26607e-2d51-4c7a-a72d-f2f1bad6ed1c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:51.685033 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 67/135] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:49:51 2022] PUT /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af/users/3c37098f90fe47ab88491e90626539d9/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 17:49:51.686468 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f04b5-0595-4757-becc-3250464ffd55 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:51.686762 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f04b5-0595-4757-becc-3250464ffd55 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:51.687052 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f04b5-0595-4757-becc-3250464ffd55 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:51.688338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 68/136] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:51.696261 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94caac-0c96-4cec-afd5-ecfde6cc4707 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:51.696740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94caac-0c96-4cec-afd5-ecfde6cc4707 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:51.696843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94caac-0c96-4cec-afd5-ecfde6cc4707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:51.736904 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ae94caac-0c96-4cec-afd5-ecfde6cc4707 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:51.738128 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ae94caac-0c96-4cec-afd5-ecfde6cc4707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:51.741222 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c10d43-14de-4195-b8d3-44b997a7571c None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:51.807927 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2ad07-e164-4725-a2f2-a328812206e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:51.809185 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2ad07-e164-4725-a2f2-a328812206e5 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:51.809738 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2ad07-e164-4725-a2f2-a328812206e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:51.810244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2ad07-e164-4725-a2f2-a328812206e5 None admin] PATH_INFO: `/v3/projects/94a321d75caa43cc94d0231c3442718f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:51.811708 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2ad07-e164-4725-a2f2-a328812206e5 None admin] RBAC: Authorizing `identity:get_project(project_id=94a321d75caa43cc94d0231c3442718f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:51.813493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2ad07-e164-4725-a2f2-a328812206e5 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:51.815528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 69/137] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:49:51 2022] GET /identity/v3/projects/94a321d75caa43cc94d0231c3442718f => generated 312 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:51.823034 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2ad07-e164-4725-a2f2-a328812206e5 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:51.868862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:51.908374 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 69/138] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:49:51.996222 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2ad07-e164-4725-a2f2-a328812206e5 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:51.998128 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94caac-0c96-4cec-afd5-ecfde6cc4707 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:52.064684 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4aec72-e355-4ef9-bd31-91c207e598c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:52.065920 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4aec72-e355-4ef9-bd31-91c207e598c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:52.066432 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4aec72-e355-4ef9-bd31-91c207e598c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:52.066944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4aec72-e355-4ef9-bd31-91c207e598c4 None admin] PATH_INFO: `/v3/projects/94a321d75caa43cc94d0231c3442718f/users/95b4e2fbe1aa4fa5b7cebed5b01639cd/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:52.069820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4aec72-e355-4ef9-bd31-91c207e598c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=94a321d75caa43cc94d0231c3442718f, user_id=95b4e2fbe1aa4fa5b7cebed5b01639cd, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:52.071628 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4aec72-e355-4ef9-bd31-91c207e598c4 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:52.094349 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 70/139] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:49:51 2022] PUT /identity/v3/projects/94a321d75caa43cc94d0231c3442718f/users/95b4e2fbe1aa4fa5b7cebed5b01639cd/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:49:52.131780 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:52.236512 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94caac-0c96-4cec-afd5-ecfde6cc4707 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:52.282886 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430aeb6-81d1-4006-8c48-dd2a9b0a7db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:52.283693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430aeb6-81d1-4006-8c48-dd2a9b0a7db3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:52.283976 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430aeb6-81d1-4006-8c48-dd2a9b0a7db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:52.284253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430aeb6-81d1-4006-8c48-dd2a9b0a7db3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:52.285449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430aeb6-81d1-4006-8c48-dd2a9b0a7db3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:52.287635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430aeb6-81d1-4006-8c48-dd2a9b0a7db3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:52.289286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 70/140] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:49:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:49:52.296056 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4aec72-e355-4ef9-bd31-91c207e598c4 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:52.406635 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:52.430368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4aec72-e355-4ef9-bd31-91c207e598c4 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:52.474900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afded4df-4fa2-4568-9403-f9f87740d5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:52.475597 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afded4df-4fa2-4568-9403-f9f87740d5a3 None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:52.475890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afded4df-4fa2-4568-9403-f9f87740d5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:52.476175 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afded4df-4fa2-4568-9403-f9f87740d5a3 None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:52.476906 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afded4df-4fa2-4568-9403-f9f87740d5a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:52.478475 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afded4df-4fa2-4568-9403-f9f87740d5a3 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:52.507349 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 71/141] 10.222.0.76 () {60 vars in 1132 bytes} [Thu May 26 17:49:52 2022] POST /identity/v3/users => generated 283 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:49:53.343923 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8eb00-5ee9-4b8b-b2cf-b367c2f3e39b None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:53.344420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8eb00-5ee9-4b8b-b2cf-b367c2f3e39b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:53.344528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8eb00-5ee9-4b8b-b2cf-b367c2f3e39b None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:53.346143 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 71/142] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:53.352612 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086aaa4-8467-43b8-91de-b6f9cee5e5ca None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:53.353078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086aaa4-8467-43b8-91de-b6f9cee5e5ca None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:53.353158 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086aaa4-8467-43b8-91de-b6f9cee5e5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:53.374555 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-d086aaa4-8467-43b8-91de-b6f9cee5e5ca None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:53.375763 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-d086aaa4-8467-43b8-91de-b6f9cee5e5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:53.468981 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 72/143] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:53.500578 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8eb00-5ee9-4b8b-b2cf-b367c2f3e39b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:53.632364 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8eb00-5ee9-4b8b-b2cf-b367c2f3e39b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:53.668492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:53.683493 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a1f0a-ab65-4f46-abda-9ddcbb34dd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:53.684643 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a1f0a-ab65-4f46-abda-9ddcbb34dd3b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:53.685129 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a1f0a-ab65-4f46-abda-9ddcbb34dd3b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:53.685625 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a1f0a-ab65-4f46-abda-9ddcbb34dd3b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:53.699492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a1f0a-ab65-4f46-abda-9ddcbb34dd3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:49:53.708333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-7a8a1f0a-ab65-4f46-abda-9ddcbb34dd3b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:49:53.708333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:49:53.708333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:49:53.708333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:49:53.708333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:49:53.708333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:49:53.708333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:49:53.708333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:49:53.708333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:49:53.708333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:49:53.708333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:49:53.708333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:49:53.708333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:49:53.708333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:49:53.708333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:49:53.708333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:49:53.709300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:49:53.709300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:49:53.709300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:49:53.709300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:49:53.709300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:49:53.709300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:49:53.709300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:49:53.709300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:49:53.709300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:49:53.709300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:49:53.709300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:49:53.709300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:49:53.709300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:49:53.709300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:49:53.709300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:49:53.709300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:49:53.710109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:49:53.710109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:49:53.710109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:49:53.710109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:49:53.710109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:49:53.710109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:49:53.710109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:49:53.710109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:49:53.710109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:49:53.710109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:49:53.710109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:49:53.710109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:49:53.712657 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 72/144] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:49:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:49:53.718758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086aaa4-8467-43b8-91de-b6f9cee5e5ca None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:53.815582 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:53.845906 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67d09a-320c-4bc7-8ced-c5278dc66af9 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:53.846703 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67d09a-320c-4bc7-8ced-c5278dc66af9 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:53.847187 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67d09a-320c-4bc7-8ced-c5278dc66af9 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:53.849980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 73/145] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:49:53.855866 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78271c5-3135-4f4d-a738-f7490a4fcfd3 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:53.856431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78271c5-3135-4f4d-a738-f7490a4fcfd3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:53.856950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78271c5-3135-4f4d-a738-f7490a4fcfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:53.857325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086aaa4-8467-43b8-91de-b6f9cee5e5ca None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:53.891443 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b78271c5-3135-4f4d-a738-f7490a4fcfd3 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:53.893062 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b78271c5-3135-4f4d-a738-f7490a4fcfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:53.898510 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b238350-e395-4fe0-9508-0b9882b30417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:53.899525 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b238350-e395-4fe0-9508-0b9882b30417 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:53.904532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b238350-e395-4fe0-9508-0b9882b30417 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:53.905062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b238350-e395-4fe0-9508-0b9882b30417 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:53.906079 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b238350-e395-4fe0-9508-0b9882b30417 None admin] RBAC: Adding query filter params (name=member) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:53.906642 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b238350-e395-4fe0-9508-0b9882b30417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:53.908730 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b238350-e395-4fe0-9508-0b9882b30417 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:53.932235 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 73/146] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:49:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:53.960612 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b238350-e395-4fe0-9508-0b9882b30417 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:54.016603 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 74/147] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:54.078743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78271c5-3135-4f4d-a738-f7490a4fcfd3 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:54.199425 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b238350-e395-4fe0-9508-0b9882b30417 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:54.249179 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78271c5-3135-4f4d-a738-f7490a4fcfd3 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:54.279290 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:54.283108 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:54.298926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96719d63-7215-474c-a9c2-b265c95427a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:54.300274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96719d63-7215-474c-a9c2-b265c95427a9 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:54.300780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96719d63-7215-474c-a9c2-b265c95427a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:54.301786 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96719d63-7215-474c-a9c2-b265c95427a9 None admin] PATH_INFO: `/v3/users/95b4e2fbe1aa4fa5b7cebed5b01639cd` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:54.303543 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96719d63-7215-474c-a9c2-b265c95427a9 None admin] RBAC: Authorizing `identity:get_user(user_id=95b4e2fbe1aa4fa5b7cebed5b01639cd)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:54.305366 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96719d63-7215-474c-a9c2-b265c95427a9 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:54.305490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45be3f7-1eba-441d-941d-79911e26f28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:54.306309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45be3f7-1eba-441d-941d-79911e26f28a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:54.306601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45be3f7-1eba-441d-941d-79911e26f28a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:54.306882 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45be3f7-1eba-441d-941d-79911e26f28a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:54.308165 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 74/148] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:49:53 2022] GET /identity/v3/users/95b4e2fbe1aa4fa5b7cebed5b01639cd => generated 275 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:49:54.313808 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96719d63-7215-474c-a9c2-b265c95427a9 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:54.324359 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45be3f7-1eba-441d-941d-79911e26f28a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:49:54.340297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-e45be3f7-1eba-441d-941d-79911e26f28a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:49:54.340297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:49:54.340297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:49:54.340297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:49:54.340297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:49:54.340297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:49:54.340297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:49:54.340297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:49:54.340297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:49:54.340297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:49:54.340297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:49:54.340297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:49:54.340297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:49:54.340297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:49:54.340297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:49:54.340297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:49:54.341203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:49:54.341203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:49:54.341203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:49:54.341203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:49:54.341203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:49:54.341203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:49:54.341203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:49:54.341203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:49:54.341203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:49:54.341203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:49:54.341203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:49:54.341203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:49:54.341203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:49:54.341203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:49:54.341203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:49:54.341203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:49:54.341979 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:49:54.341979 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:49:54.341979 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:49:54.341979 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:49:54.341979 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:49:54.341979 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:49:54.341979 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:49:54.341979 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:49:54.341979 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:49:54.341979 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:49:54.341979 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:49:54.341979 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:49:54.342693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 75/149] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:49:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:49:54.354651 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45be3f7-1eba-441d-941d-79911e26f28a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:54.473964 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45be3f7-1eba-441d-941d-79911e26f28a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:54.513396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96719d63-7215-474c-a9c2-b265c95427a9 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:54.534359 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3331f9-6b0b-44ee-a44a-1dccbcc96d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:54.535050 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3331f9-6b0b-44ee-a44a-1dccbcc96d3e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:54.535335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3331f9-6b0b-44ee-a44a-1dccbcc96d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:54.536284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3331f9-6b0b-44ee-a44a-1dccbcc96d3e None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:54.537474 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3331f9-6b0b-44ee-a44a-1dccbcc96d3e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:54.537783 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3331f9-6b0b-44ee-a44a-1dccbcc96d3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:54.539338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3331f9-6b0b-44ee-a44a-1dccbcc96d3e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:54.562225 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 76/150] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:49:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:54.567284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3331f9-6b0b-44ee-a44a-1dccbcc96d3e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:54.604048 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3591c6-af3c-4275-800a-1ac89d475670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:54.604640 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3591c6-af3c-4275-800a-1ac89d475670 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:54.604923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3591c6-af3c-4275-800a-1ac89d475670 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:54.605196 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3591c6-af3c-4275-800a-1ac89d475670 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:54.628872 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:54.706553 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3331f9-6b0b-44ee-a44a-1dccbcc96d3e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:54.715904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:54.725182 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3591c6-af3c-4275-800a-1ac89d475670 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:54.726708 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3591c6-af3c-4275-800a-1ac89d475670 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:54.762714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442019f-e79f-4031-aba3-9ca1f1a22bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:54.763714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442019f-e79f-4031-aba3-9ca1f1a22bbf None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:54.764002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442019f-e79f-4031-aba3-9ca1f1a22bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:54.764381 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442019f-e79f-4031-aba3-9ca1f1a22bbf None admin] PATH_INFO: `/v3/users/b087f9732f764f9a9f4c915abe5814c8` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:54.771117 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442019f-e79f-4031-aba3-9ca1f1a22bbf None admin] RBAC: Authorizing `identity:get_user(user_id=b087f9732f764f9a9f4c915abe5814c8)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:54.772869 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442019f-e79f-4031-aba3-9ca1f1a22bbf None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:54.774837 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 77/151] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:49:54 2022] GET /identity/v3/users/b087f9732f764f9a9f4c915abe5814c8 => generated 289 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:54.782233 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442019f-e79f-4031-aba3-9ca1f1a22bbf None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:54.890770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 75/152] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:49:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 17:49:54.892223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82969f74-546f-4bde-9361-bdaa2cd282a4 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:54.892511 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82969f74-546f-4bde-9361-bdaa2cd282a4 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:54.892788 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82969f74-546f-4bde-9361-bdaa2cd282a4 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:54.897119 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 76/153] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:54.898013 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82969f74-546f-4bde-9361-bdaa2cd282a4 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:55.016346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442019f-e79f-4031-aba3-9ca1f1a22bbf None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:55.034588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:55.097875 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14355e-95dc-44d9-a9cd-f26f8c4eb246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:55.098855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14355e-95dc-44d9-a9cd-f26f8c4eb246 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:55.099230 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14355e-95dc-44d9-a9cd-f26f8c4eb246 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:55.099509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14355e-95dc-44d9-a9cd-f26f8c4eb246 None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:55.103005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14355e-95dc-44d9-a9cd-f26f8c4eb246 None admin] RBAC: Authorizing `identity:get_project(project_id=89f44d18dc9b464faeb1bf0bd03569af)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:55.104595 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14355e-95dc-44d9-a9cd-f26f8c4eb246 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:55.106505 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 78/154] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:49:54 2022] GET /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af => generated 298 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:49:55.108872 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864379c4-ba8e-42a0-8e90-74aabf0f136d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:55.109290 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864379c4-ba8e-42a0-8e90-74aabf0f136d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:55.109575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864379c4-ba8e-42a0-8e90-74aabf0f136d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:55.141785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-864379c4-ba8e-42a0-8e90-74aabf0f136d None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:55.145170 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-864379c4-ba8e-42a0-8e90-74aabf0f136d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:55.165478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:55.175385 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82969f74-546f-4bde-9361-bdaa2cd282a4 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:55.242133 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93605d9a-ede3-4008-a1f6-b1c4c1fab948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:55.243123 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93605d9a-ede3-4008-a1f6-b1c4c1fab948 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:55.243408 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93605d9a-ede3-4008-a1f6-b1c4c1fab948 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:55.243680 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93605d9a-ede3-4008-a1f6-b1c4c1fab948 None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:55.251150 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93605d9a-ede3-4008-a1f6-b1c4c1fab948 None admin] RBAC: Authorizing `identity:get_project(project_id=89f44d18dc9b464faeb1bf0bd03569af)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:55.252653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93605d9a-ede3-4008-a1f6-b1c4c1fab948 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:55.255592 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 77/155] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:49:54 2022] GET /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af => generated 298 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:55.256449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93605d9a-ede3-4008-a1f6-b1c4c1fab948 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:55.311235 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 79/156] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:55.312808 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d3c65-8987-46cf-8d3e-b9df38ef83d6 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:55.313102 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d3c65-8987-46cf-8d3e-b9df38ef83d6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:55.313376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d3c65-8987-46cf-8d3e-b9df38ef83d6 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:55.315387 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 80/157] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:55.316309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d3c65-8987-46cf-8d3e-b9df38ef83d6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:55.478445 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93605d9a-ede3-4008-a1f6-b1c4c1fab948 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:55.558953 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d3c65-8987-46cf-8d3e-b9df38ef83d6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:55.570240 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:55.579800 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad3cea-e008-4cb4-9a55-7d8c5af8fa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:55.580727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad3cea-e008-4cb4-9a55-7d8c5af8fa75 None admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:55.581013 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad3cea-e008-4cb4-9a55-7d8c5af8fa75 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:55.581291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad3cea-e008-4cb4-9a55-7d8c5af8fa75 None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af/users/b087f9732f764f9a9f4c915abe5814c8/roles/5b3eae4902f44cc9a56bf2ec2a8e13f5` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:55.591307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad3cea-e008-4cb4-9a55-7d8c5af8fa75 None admin] RBAC: Authorizing `identity:create_grant(project_id=89f44d18dc9b464faeb1bf0bd03569af, user_id=b087f9732f764f9a9f4c915abe5814c8, role_id=5b3eae4902f44cc9a56bf2ec2a8e13f5)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:55.592868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad3cea-e008-4cb4-9a55-7d8c5af8fa75 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:55.624510 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 78/158] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:49:55 2022] PUT /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af/users/b087f9732f764f9a9f4c915abe5814c8/roles/5b3eae4902f44cc9a56bf2ec2a8e13f5 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:49:55.626134 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96841c-f080-49b2-bc8f-589cde07ce3d None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:55.626777 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96841c-f080-49b2-bc8f-589cde07ce3d None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:55.627322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96841c-f080-49b2-bc8f-589cde07ce3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:55.633630 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265053b9-cdbc-4cb9-a69a-0a365b88c745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:55.634683 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265053b9-cdbc-4cb9-a69a-0a365b88c745 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:55.635170 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265053b9-cdbc-4cb9-a69a-0a365b88c745 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:55.635657 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265053b9-cdbc-4cb9-a69a-0a365b88c745 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:55.636808 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265053b9-cdbc-4cb9-a69a-0a365b88c745 None admin] RBAC: Adding query filter params (role.id=eb1858a5621543028836223abcb3c0a1, scope.project.id=89f44d18dc9b464faeb1bf0bd03569af, user.id=95b4e2fbe1aa4fa5b7cebed5b01639cd) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:55.637336 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265053b9-cdbc-4cb9-a69a-0a365b88c745 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:55.639116 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265053b9-cdbc-4cb9-a69a-0a365b88c745 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:55.651419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 81/159] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:49:55 2022] GET /identity/v3/role_assignments?user.id=95b4e2fbe1aa4fa5b7cebed5b01639cd&scope.project.id=89f44d18dc9b464faeb1bf0bd03569af&role.id=eb1858a5621543028836223abcb3c0a1 => generated 608 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:55.652600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265053b9-cdbc-4cb9-a69a-0a365b88c745 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:55.659370 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-3f96841c-f080-49b2-bc8f-589cde07ce3d None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:55.660771 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-3f96841c-f080-49b2-bc8f-589cde07ce3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:55.710029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:55.812794 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 79/160] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:55.862679 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96841c-f080-49b2-bc8f-589cde07ce3d None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:55.905611 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265053b9-cdbc-4cb9-a69a-0a365b88c745 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:56.007732 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105e82f-0ab8-4c68-87a6-69d7dd454e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:56.008560 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105e82f-0ab8-4c68-87a6-69d7dd454e72 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:56.008843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105e82f-0ab8-4c68-87a6-69d7dd454e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:56.009115 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105e82f-0ab8-4c68-87a6-69d7dd454e72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:56.051862 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105e82f-0ab8-4c68-87a6-69d7dd454e72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:49:56.059693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-e105e82f-0ab8-4c68-87a6-69d7dd454e72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:49:56.059693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:49:56.059693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:49:56.059693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:49:56.059693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:49:56.059693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:49:56.059693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:49:56.059693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:49:56.059693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:49:56.059693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:49:56.059693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:49:56.059693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:49:56.059693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:49:56.059693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:49:56.059693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:49:56.059693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:49:56.060695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:49:56.060695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:49:56.060695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:49:56.060695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:49:56.060695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:49:56.060695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:49:56.060695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:49:56.060695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:49:56.060695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:49:56.060695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:49:56.060695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:49:56.060695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:49:56.060695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:49:56.060695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:49:56.060695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:49:56.060695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:49:56.061892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:49:56.061892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:49:56.061892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:49:56.061892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:49:56.061892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:49:56.061892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:49:56.061892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:49:56.061892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:49:56.061892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:49:56.061892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:49:56.061892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:49:56.061892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:49:56.061892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 82/161] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:49:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:49:56.069133 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105e82f-0ab8-4c68-87a6-69d7dd454e72 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:56.076730 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96841c-f080-49b2-bc8f-589cde07ce3d None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:56.152472 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:56.169232 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ed29a-b806-4d25-95eb-17a5e4f25713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:56.170067 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ed29a-b806-4d25-95eb-17a5e4f25713 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:56.170434 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ed29a-b806-4d25-95eb-17a5e4f25713 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:56.170711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ed29a-b806-4d25-95eb-17a5e4f25713 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:56.205677 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ed29a-b806-4d25-95eb-17a5e4f25713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:49:56.214820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-f92ed29a-b806-4d25-95eb-17a5e4f25713 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:49:56.214820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:49:56.214820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:49:56.214820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:49:56.214820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:49:56.214820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:49:56.214820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:49:56.214820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:49:56.214820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:49:56.214820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:49:56.214820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:49:56.214820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:49:56.214820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:49:56.214820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:49:56.214820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:49:56.214820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:49:56.215819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:49:56.215819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:49:56.215819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:49:56.215819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:49:56.215819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:49:56.215819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:49:56.215819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:49:56.215819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:49:56.215819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:49:56.215819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:49:56.215819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:49:56.215819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:49:56.215819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:49:56.215819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:49:56.215819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:49:56.215819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:49:56.216596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:49:56.216596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:49:56.216596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:49:56.216596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:49:56.216596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:49:56.216596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:49:56.216596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:49:56.216596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:49:56.216596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:49:56.216596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:49:56.216596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:49:56.216596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:49:56.216596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 80/162] 10.222.0.76 () {56 vars in 1114 bytes} [Thu May 26 17:49:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:49:56.224888 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ed29a-b806-4d25-95eb-17a5e4f25713 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:56.256346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105e82f-0ab8-4c68-87a6-69d7dd454e72 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:56.294592 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:56.327354 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6a992-71cb-43f2-9b2a-425fa247ff2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:56.329042 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6a992-71cb-43f2-9b2a-425fa247ff2e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:56.329333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6a992-71cb-43f2-9b2a-425fa247ff2e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:56.329612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6a992-71cb-43f2-9b2a-425fa247ff2e None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:56.330408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6a992-71cb-43f2-9b2a-425fa247ff2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:56.330764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6a992-71cb-43f2-9b2a-425fa247ff2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:56.332118 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6a992-71cb-43f2-9b2a-425fa247ff2e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:56.348990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 83/163] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:49:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:56.348990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179c8e8-b3bf-41fc-9a78-52438f1e56f1 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:56.348990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179c8e8-b3bf-41fc-9a78-52438f1e56f1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:56.348990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179c8e8-b3bf-41fc-9a78-52438f1e56f1 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:56.348990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 84/164] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:56.351651 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9ea5e-8e48-4a7b-bcce-83749ed4e91d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:56.351952 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9ea5e-8e48-4a7b-bcce-83749ed4e91d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:56.352205 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9ea5e-8e48-4a7b-bcce-83749ed4e91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:56.371035 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-23c9ea5e-8e48-4a7b-bcce-83749ed4e91d None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:56.372236 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-23c9ea5e-8e48-4a7b-bcce-83749ed4e91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:56.393505 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ed29a-b806-4d25-95eb-17a5e4f25713 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:56.450033 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 85/165] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:56.450955 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9ea5e-8e48-4a7b-bcce-83749ed4e91d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:56.452697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8361b-a50d-4365-8ac8-3c9572b38a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:56.453648 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8361b-a50d-4365-8ac8-3c9572b38a96 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:56.454153 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8361b-a50d-4365-8ac8-3c9572b38a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:56.454656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8361b-a50d-4365-8ac8-3c9572b38a96 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:56.455724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8361b-a50d-4365-8ac8-3c9572b38a96 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:56.456260 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8361b-a50d-4365-8ac8-3c9572b38a96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:56.458514 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8361b-a50d-4365-8ac8-3c9572b38a96 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:56.488462 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 81/166] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:49:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:56.490398 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8361b-a50d-4365-8ac8-3c9572b38a96 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:56.514560 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:56.645414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:56.662954 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9ea5e-8e48-4a7b-bcce-83749ed4e91d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:56.683696 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8361b-a50d-4365-8ac8-3c9572b38a96 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:56.700556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd70bd-8fa0-4f4f-8435-60f9a50473ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:56.701809 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd70bd-8fa0-4f4f-8435-60f9a50473ee None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:56.702364 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd70bd-8fa0-4f4f-8435-60f9a50473ee None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:56.702837 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd70bd-8fa0-4f4f-8435-60f9a50473ee None admin] PATH_INFO: `/v3/users/3c37098f90fe47ab88491e90626539d9` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:56.704263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd70bd-8fa0-4f4f-8435-60f9a50473ee None admin] RBAC: Authorizing `identity:get_user(user_id=3c37098f90fe47ab88491e90626539d9)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:56.706626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd70bd-8fa0-4f4f-8435-60f9a50473ee None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:56.709545 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 86/167] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:49:56 2022] GET /identity/v3/users/3c37098f90fe47ab88491e90626539d9 => generated 247 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:56.710634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd70bd-8fa0-4f4f-8435-60f9a50473ee None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:56.759154 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c8100-53fe-4ae1-adf9-4074ac6b5248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:56.760089 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c8100-53fe-4ae1-adf9-4074ac6b5248 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:56.760350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c8100-53fe-4ae1-adf9-4074ac6b5248 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:56.760662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c8100-53fe-4ae1-adf9-4074ac6b5248 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:56.788194 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860c8100-53fe-4ae1-adf9-4074ac6b5248 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:49:56.808611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-860c8100-53fe-4ae1-adf9-4074ac6b5248 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:49:56.808611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:49:56.808611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:49:56.808611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:49:56.808611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:49:56.808611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:49:56.808611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:49:56.808611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:49:56.808611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:49:56.808611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:49:56.808611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:49:56.808611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:49:56.808611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:49:56.808611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:49:56.808611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:49:56.808611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:49:56.809566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:49:56.809566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:49:56.809566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:49:56.809566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:49:56.809566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:49:56.809566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:49:56.809566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:49:56.809566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:49:56.809566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:49:56.809566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:49:56.809566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:49:56.809566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:49:56.809566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:49:56.809566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:49:56.809566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:49:56.809566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:49:56.811137 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:49:56.811137 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:49:56.811137 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:49:56.811137 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:49:56.811137 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:49:56.811137 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:49:56.811137 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:49:56.811137 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:49:56.811137 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:49:56.811137 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:49:56.811137 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:49:56.811137 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:49:56.811137 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 82/168] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:49:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:49:56.811137 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c8100-53fe-4ae1-adf9-4074ac6b5248 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:56.822569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:56.832490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd70bd-8fa0-4f4f-8435-60f9a50473ee None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:56.870170 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a244fe0-33f6-4cd2-95d7-ae2fd407c72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:56.871478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a244fe0-33f6-4cd2-95d7-ae2fd407c72c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:56.871962 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a244fe0-33f6-4cd2-95d7-ae2fd407c72c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:56.872458 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a244fe0-33f6-4cd2-95d7-ae2fd407c72c None admin] PATH_INFO: `/v3/users/95b4e2fbe1aa4fa5b7cebed5b01639cd` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:56.873919 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a244fe0-33f6-4cd2-95d7-ae2fd407c72c None admin] RBAC: Authorizing `identity:get_user(user_id=95b4e2fbe1aa4fa5b7cebed5b01639cd)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:56.875699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a244fe0-33f6-4cd2-95d7-ae2fd407c72c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:56.877621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 87/169] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:49:56 2022] GET /identity/v3/users/95b4e2fbe1aa4fa5b7cebed5b01639cd => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:56.878665 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a244fe0-33f6-4cd2-95d7-ae2fd407c72c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:56.997943 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c8100-53fe-4ae1-adf9-4074ac6b5248 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:57.043003 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c92e7-dc4f-4e9d-aa7c-2a8355076ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:57.044537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c92e7-dc4f-4e9d-aa7c-2a8355076ddb None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:57.045151 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c92e7-dc4f-4e9d-aa7c-2a8355076ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:57.045638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c92e7-dc4f-4e9d-aa7c-2a8355076ddb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:57.070576 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:57.168331 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c92e7-dc4f-4e9d-aa7c-2a8355076ddb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:57.169843 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c92e7-dc4f-4e9d-aa7c-2a8355076ddb None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:57.324429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a244fe0-33f6-4cd2-95d7-ae2fd407c72c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:57.339990 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 83/170] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:49:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:49:57.341261 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c92e7-dc4f-4e9d-aa7c-2a8355076ddb None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:57.424624 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:57.444399 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4ba98-7585-40f5-93c3-0041aacf687d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:57.445087 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4ba98-7585-40f5-93c3-0041aacf687d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:57.445371 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4ba98-7585-40f5-93c3-0041aacf687d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:57.445644 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4ba98-7585-40f5-93c3-0041aacf687d None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:57.446432 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4ba98-7585-40f5-93c3-0041aacf687d None admin] RBAC: Adding query filter params (name=member) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:57.446736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4ba98-7585-40f5-93c3-0041aacf687d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:57.448156 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4ba98-7585-40f5-93c3-0041aacf687d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:57.465591 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 88/171] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:49:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 587 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:57.466488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4ba98-7585-40f5-93c3-0041aacf687d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:57.526881 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:57.569162 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c92e7-dc4f-4e9d-aa7c-2a8355076ddb None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:57.638085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c23ccd-2d57-4917-b9ab-999d57196b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:57.638669 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c23ccd-2d57-4917-b9ab-999d57196b08 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:57.638954 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c23ccd-2d57-4917-b9ab-999d57196b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:57.639230 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c23ccd-2d57-4917-b9ab-999d57196b08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:57.695953 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4ba98-7585-40f5-93c3-0041aacf687d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:57.758159 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637bc18-8e7c-4954-9f5a-a2f421784a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:57.759050 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637bc18-8e7c-4954-9f5a-a2f421784a1d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:57.759330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637bc18-8e7c-4954-9f5a-a2f421784a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:57.759604 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637bc18-8e7c-4954-9f5a-a2f421784a1d None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:57.763231 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637bc18-8e7c-4954-9f5a-a2f421784a1d None admin] RBAC: Authorizing `identity:get_project(project_id=89f44d18dc9b464faeb1bf0bd03569af)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:57.764710 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637bc18-8e7c-4954-9f5a-a2f421784a1d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:57.766489 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 89/172] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:49:57 2022] GET /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af => generated 298 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:57.767293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637bc18-8e7c-4954-9f5a-a2f421784a1d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:57.774339 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c23ccd-2d57-4917-b9ab-999d57196b08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:57.775893 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c23ccd-2d57-4917-b9ab-999d57196b08 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:57.858561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:57.928799 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:57.950609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 84/173] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:49:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:49:57.951741 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637bc18-8e7c-4954-9f5a-a2f421784a1d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:57.952234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe989994-e4e1-4666-bb7b-92a77f70ce41 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:57.952695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe989994-e4e1-4666-bb7b-92a77f70ce41 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:57.953078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe989994-e4e1-4666-bb7b-92a77f70ce41 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:57.955301 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 85/174] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:57.956165 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe989994-e4e1-4666-bb7b-92a77f70ce41 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:58.004661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0ea42-fc71-4ad1-990e-908ef698d06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:58.006172 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0ea42-fc71-4ad1-990e-908ef698d06a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:58.006996 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0ea42-fc71-4ad1-990e-908ef698d06a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:58.007481 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0ea42-fc71-4ad1-990e-908ef698d06a None admin] PATH_INFO: `/v3/users/95b4e2fbe1aa4fa5b7cebed5b01639cd` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:58.008976 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0ea42-fc71-4ad1-990e-908ef698d06a None admin] RBAC: Authorizing `identity:get_user(user_id=95b4e2fbe1aa4fa5b7cebed5b01639cd)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:58.011943 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0ea42-fc71-4ad1-990e-908ef698d06a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:58.014003 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 90/175] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:49:57 2022] GET /identity/v3/users/95b4e2fbe1aa4fa5b7cebed5b01639cd => generated 275 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:58.015544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c360afe-b53b-4406-8adf-c14fccc47b9f None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:58.016048 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c360afe-b53b-4406-8adf-c14fccc47b9f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:58.017275 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c360afe-b53b-4406-8adf-c14fccc47b9f None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:58.018937 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 91/176] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:58.022808 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c360afe-b53b-4406-8adf-c14fccc47b9f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:58.096569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe989994-e4e1-4666-bb7b-92a77f70ce41 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:58.144906 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057169e-81d9-496c-b634-a1ba304dcc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:58.145880 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057169e-81d9-496c-b634-a1ba304dcc1c None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:58.146169 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057169e-81d9-496c-b634-a1ba304dcc1c None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:58.146492 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057169e-81d9-496c-b634-a1ba304dcc1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:58.147353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057169e-81d9-496c-b634-a1ba304dcc1c None admin] RBAC: Adding query filter params (role.id=4fa8f06085c741a5b3cf64e63cc0754f, scope.project.id=89f44d18dc9b464faeb1bf0bd03569af, user.id=3c37098f90fe47ab88491e90626539d9) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:58.147675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057169e-81d9-496c-b634-a1ba304dcc1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:58.149141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057169e-81d9-496c-b634-a1ba304dcc1c None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:58.153957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c360afe-b53b-4406-8adf-c14fccc47b9f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:58.158732 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 86/177] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:49:57 2022] GET /identity/v3/role_assignments?user.id=3c37098f90fe47ab88491e90626539d9&scope.project.id=89f44d18dc9b464faeb1bf0bd03569af&role.id=4fa8f06085c741a5b3cf64e63cc0754f => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:58.160173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50305c-1afd-4b2e-890a-18b9d2b92e81 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:58.160744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50305c-1afd-4b2e-890a-18b9d2b92e81 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:58.161158 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50305c-1afd-4b2e-890a-18b9d2b92e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:58.167916 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:58.180386 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-7f50305c-1afd-4b2e-890a-18b9d2b92e81 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:58.181860 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-7f50305c-1afd-4b2e-890a-18b9d2b92e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:58.213441 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb033198-e7fd-4435-a6c7-5cc1e18edf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:58.214814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb033198-e7fd-4435-a6c7-5cc1e18edf38 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:58.215316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb033198-e7fd-4435-a6c7-5cc1e18edf38 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:58.215879 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb033198-e7fd-4435-a6c7-5cc1e18edf38 None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:58.217449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb033198-e7fd-4435-a6c7-5cc1e18edf38 None admin] RBAC: Authorizing `identity:get_project(project_id=89f44d18dc9b464faeb1bf0bd03569af)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:58.219317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb033198-e7fd-4435-a6c7-5cc1e18edf38 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:58.221574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 92/178] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:49:58 2022] GET /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:58.222653 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb033198-e7fd-4435-a6c7-5cc1e18edf38 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:58.264940 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:58.273428 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 87/179] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:58.275560 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16eb79-e8d7-402d-bda1-1345ad6c6e5c None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:58.276099 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16eb79-e8d7-402d-bda1-1345ad6c6e5c None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:58.276579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16eb79-e8d7-402d-bda1-1345ad6c6e5c None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:58.282622 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 88/180] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:58.284118 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e4b6e-0fa9-4899-a065-e20be4686339 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:58.284611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e4b6e-0fa9-4899-a065-e20be4686339 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:58.285149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e4b6e-0fa9-4899-a065-e20be4686339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:58.305969 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-733e4b6e-0fa9-4899-a065-e20be4686339 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:58.307622 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-733e4b6e-0fa9-4899-a065-e20be4686339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:58.372956 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb033198-e7fd-4435-a6c7-5cc1e18edf38 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:58.400099 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 89/181] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:58.401688 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e4b6e-0fa9-4899-a065-e20be4686339 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:58.421241 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b866516f-4c2d-431e-9a2d-559656f0efc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:58.421905 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b866516f-4c2d-431e-9a2d-559656f0efc6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:58.422205 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b866516f-4c2d-431e-9a2d-559656f0efc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:58.422492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b866516f-4c2d-431e-9a2d-559656f0efc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:58.490787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:58.516632 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b866516f-4c2d-431e-9a2d-559656f0efc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:58.518179 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b866516f-4c2d-431e-9a2d-559656f0efc6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:58.542993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e4b6e-0fa9-4899-a065-e20be4686339 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:58.555247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:58.583258 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4191b80-41e7-4c2e-b388-9173c4f84d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:58.584107 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4191b80-41e7-4c2e-b388-9173c4f84d72 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:58.584291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4191b80-41e7-4c2e-b388-9173c4f84d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:58.584575 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4191b80-41e7-4c2e-b388-9173c4f84d72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:58.585425 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4191b80-41e7-4c2e-b388-9173c4f84d72 None admin] RBAC: Adding query filter params (role.id=e731efeb1d3c407fb58cf64a02d95796, scope.project.id=89f44d18dc9b464faeb1bf0bd03569af, user.id=95b4e2fbe1aa4fa5b7cebed5b01639cd) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:58.585740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4191b80-41e7-4c2e-b388-9173c4f84d72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:58.587243 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4191b80-41e7-4c2e-b388-9173c4f84d72 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:58.598223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 90/182] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:49:58 2022] GET /identity/v3/role_assignments?user.id=95b4e2fbe1aa4fa5b7cebed5b01639cd&scope.project.id=89f44d18dc9b464faeb1bf0bd03569af&role.id=e731efeb1d3c407fb58cf64a02d95796 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:58.598223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b2dac-4fd9-44c6-ab68-091d8ea9979a None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:58.598223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b2dac-4fd9-44c6-ab68-091d8ea9979a None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:58.598223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b2dac-4fd9-44c6-ab68-091d8ea9979a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:58.619260 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-031b2dac-4fd9-44c6-ab68-091d8ea9979a None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:58.620504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-031b2dac-4fd9-44c6-ab68-091d8ea9979a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:58.654047 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 93/183] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:49:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:49:58.655308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b866516f-4c2d-431e-9a2d-559656f0efc6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:58.717467 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 91/184] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:58.718859 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b2dac-4fd9-44c6-ab68-091d8ea9979a None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:58.791370 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:58.805864 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b866516f-4c2d-431e-9a2d-559656f0efc6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:58.843242 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:58.856280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b2dac-4fd9-44c6-ab68-091d8ea9979a None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:58.869227 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f64945-4d84-48a1-a319-8d96e3faf263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:58.870043 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f64945-4d84-48a1-a319-8d96e3faf263 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:58.870399 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f64945-4d84-48a1-a319-8d96e3faf263 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:58.870679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f64945-4d84-48a1-a319-8d96e3faf263 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:58.885162 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f64945-4d84-48a1-a319-8d96e3faf263 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:49:58.893386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-51f64945-4d84-48a1-a319-8d96e3faf263 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:49:58.893386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:49:58.893386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:49:58.893386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:49:58.893386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:49:58.893386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:49:58.893386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:49:58.893386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:49:58.893386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:49:58.893386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:49:58.893386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:49:58.893386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:49:58.893386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:49:58.893386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:49:58.893386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:49:58.893386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:49:58.895237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:49:58.895237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:49:58.895237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:49:58.895237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:49:58.895237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:49:58.895237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:49:58.895237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:49:58.895237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:49:58.895237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:49:58.895237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:49:58.895237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:49:58.895237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:49:58.895237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:49:58.895237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:49:58.895237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:49:58.895237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:49:58.896068 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:49:58.896068 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:49:58.896068 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:49:58.896068 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:49:58.896068 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:49:58.896068 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:49:58.896068 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:49:58.896068 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:49:58.896068 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:49:58.896068 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:49:58.896068 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:49:58.896068 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:49:58.896682 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 94/185] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:49:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:49:58.898261 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f64945-4d84-48a1-a319-8d96e3faf263 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:58.914733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a5c48-86c3-42ca-9d5f-64269b8b6b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:58.915575 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a5c48-86c3-42ca-9d5f-64269b8b6b8e None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:58.915865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a5c48-86c3-42ca-9d5f-64269b8b6b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:58.916563 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a5c48-86c3-42ca-9d5f-64269b8b6b8e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:58.930123 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a5c48-86c3-42ca-9d5f-64269b8b6b8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:49:58.939403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-3c9a5c48-86c3-42ca-9d5f-64269b8b6b8e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:49:58.939403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:49:58.939403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:49:58.939403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:49:58.939403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:49:58.939403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:49:58.939403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:49:58.939403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:49:58.939403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:49:58.939403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:49:58.939403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:49:58.939403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:49:58.939403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:49:58.939403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:49:58.939403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:49:58.939403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:49:58.940353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:49:58.940353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:49:58.940353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:49:58.940353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:49:58.940353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:49:58.940353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:49:58.940353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:49:58.940353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:49:58.940353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:49:58.940353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:49:58.940353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:49:58.940353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:49:58.940353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:49:58.940353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:49:58.940353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:49:58.940353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:49:58.941176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:49:58.941176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:49:58.941176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:49:58.941176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:49:58.941176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:49:58.941176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:49:58.941176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:49:58.941176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:49:58.941176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:49:58.941176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:49:58.941176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:49:58.941176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:49:58.941176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 92/186] 10.222.0.76 () {56 vars in 1114 bytes} [Thu May 26 17:49:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:49:58.942306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a5c48-86c3-42ca-9d5f-64269b8b6b8e None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:59.026418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f64945-4d84-48a1-a319-8d96e3faf263 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:59.060378 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a5c48-86c3-42ca-9d5f-64269b8b6b8e None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:59.061962 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe05df9-a7bc-4295-ad71-7bf7e0dde1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:59.062860 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe05df9-a7bc-4295-ad71-7bf7e0dde1d3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:59.063145 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe05df9-a7bc-4295-ad71-7bf7e0dde1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:59.063422 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe05df9-a7bc-4295-ad71-7bf7e0dde1d3 None admin] PATH_INFO: `/v3/projects/94a321d75caa43cc94d0231c3442718f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:59.064628 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe05df9-a7bc-4295-ad71-7bf7e0dde1d3 None admin] RBAC: Authorizing `identity:get_project(project_id=94a321d75caa43cc94d0231c3442718f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:59.066109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe05df9-a7bc-4295-ad71-7bf7e0dde1d3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:59.068417 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 95/187] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:49:58 2022] GET /identity/v3/projects/94a321d75caa43cc94d0231c3442718f => generated 312 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:59.069205 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe05df9-a7bc-4295-ad71-7bf7e0dde1d3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:59.078584 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:59.096558 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60800de-8fa0-4f66-8da2-cd69390944e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:59.097357 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60800de-8fa0-4f66-8da2-cd69390944e6 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:59.097652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60800de-8fa0-4f66-8da2-cd69390944e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:59.098090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60800de-8fa0-4f66-8da2-cd69390944e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:59.099316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60800de-8fa0-4f66-8da2-cd69390944e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:59.100798 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60800de-8fa0-4f66-8da2-cd69390944e6 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:59.102659 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 93/188] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:49:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:59.103253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60800de-8fa0-4f66-8da2-cd69390944e6 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:59.127894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:59.179081 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe05df9-a7bc-4295-ad71-7bf7e0dde1d3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:59.206984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60800de-8fa0-4f66-8da2-cd69390944e6 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:59.213290 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823a4d3-7d04-49bc-bf88-98bdf0c2b1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:59.213970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823a4d3-7d04-49bc-bf88-98bdf0c2b1fe None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:59.214286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823a4d3-7d04-49bc-bf88-98bdf0c2b1fe None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:59.214567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823a4d3-7d04-49bc-bf88-98bdf0c2b1fe None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:59.215321 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823a4d3-7d04-49bc-bf88-98bdf0c2b1fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:59.215629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823a4d3-7d04-49bc-bf88-98bdf0c2b1fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:59.217031 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823a4d3-7d04-49bc-bf88-98bdf0c2b1fe None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:59.225743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 96/189] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:49:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:59.227061 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b8698-e46d-4a07-9439-157d4071186c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:59.227351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b8698-e46d-4a07-9439-157d4071186c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:59.227630 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b8698-e46d-4a07-9439-157d4071186c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:59.228871 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 97/190] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:49:59.230937 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b8698-e46d-4a07-9439-157d4071186c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:59.244264 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632c990-1256-40d1-a3e1-dca60f18edcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:59.244940 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632c990-1256-40d1-a3e1-dca60f18edcf None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:59.245229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632c990-1256-40d1-a3e1-dca60f18edcf None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:59.245510 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632c990-1256-40d1-a3e1-dca60f18edcf None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:59.246299 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632c990-1256-40d1-a3e1-dca60f18edcf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:59.246609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632c990-1256-40d1-a3e1-dca60f18edcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:59.247993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632c990-1256-40d1-a3e1-dca60f18edcf None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:59.256560 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 94/191] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:49:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:59.257350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632c990-1256-40d1-a3e1-dca60f18edcf None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:59.309498 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:59.334718 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b8698-e46d-4a07-9439-157d4071186c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:59.349110 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:59.361385 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632c990-1256-40d1-a3e1-dca60f18edcf None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:59.369760 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4d6af-c074-4f4b-85a1-686325e12f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:59.370458 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4d6af-c074-4f4b-85a1-686325e12f4c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:59.370743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4d6af-c074-4f4b-85a1-686325e12f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:59.371021 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4d6af-c074-4f4b-85a1-686325e12f4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:59.371851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e4d6af-c074-4f4b-85a1-686325e12f4c None admin] RBAC: Adding query filter params (role.id=eb1858a5621543028836223abcb3c0a1, scope.project.id=94a321d75caa43cc94d0231c3442718f, user.id=95b4e2fbe1aa4fa5b7cebed5b01639cd) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:49:59.372168 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e4d6af-c074-4f4b-85a1-686325e12f4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:59.373632 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e4d6af-c074-4f4b-85a1-686325e12f4c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:59.379691 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 98/192] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:49:59 2022] GET /identity/v3/role_assignments?user.id=95b4e2fbe1aa4fa5b7cebed5b01639cd&scope.project.id=94a321d75caa43cc94d0231c3442718f&role.id=eb1858a5621543028836223abcb3c0a1 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:59.380301 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4d6af-c074-4f4b-85a1-686325e12f4c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:59.405178 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd709059-682d-4e88-af87-061ddb52fdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:59.405874 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd709059-682d-4e88-af87-061ddb52fdbd None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:59.406763 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd709059-682d-4e88-af87-061ddb52fdbd None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:59.407074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd709059-682d-4e88-af87-061ddb52fdbd None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:59.407827 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd709059-682d-4e88-af87-061ddb52fdbd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:59.409339 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd709059-682d-4e88-af87-061ddb52fdbd None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:59.432664 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 95/193] 10.222.0.76 () {60 vars in 1132 bytes} [Thu May 26 17:49:59 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:49:59.433991 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cfb78-6133-4148-bc72-32020afa2f03 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:59.434310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cfb78-6133-4148-bc72-32020afa2f03 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:59.434597 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cfb78-6133-4148-bc72-32020afa2f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:59.457734 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-153cfb78-6133-4148-bc72-32020afa2f03 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:49:59.459247 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-153cfb78-6133-4148-bc72-32020afa2f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:49:59.531447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4d6af-c074-4f4b-85a1-686325e12f4c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:59.558944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 96/194] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:49:59.560155 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cfb78-6133-4148-bc72-32020afa2f03 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:59.575898 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebb1ff-2505-431a-b477-b91b79d032a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:59.576804 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebb1ff-2505-431a-b477-b91b79d032a2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:59.577225 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebb1ff-2505-431a-b477-b91b79d032a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:59.577364 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebb1ff-2505-431a-b477-b91b79d032a2 None admin] PATH_INFO: `/v3/users/cb7c2cbcc6b843458bbdbf6998920f1e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:59.594647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ebb1ff-2505-431a-b477-b91b79d032a2 None admin] RBAC: Authorizing `identity:get_user(user_id=cb7c2cbcc6b843458bbdbf6998920f1e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:59.596163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ebb1ff-2505-431a-b477-b91b79d032a2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:59.599087 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 99/195] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:49:59 2022] GET /identity/v3/users/cb7c2cbcc6b843458bbdbf6998920f1e => generated 283 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:59.603757 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:59.603949 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebb1ff-2505-431a-b477-b91b79d032a2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:59.644661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:59.691081 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cfb78-6133-4148-bc72-32020afa2f03 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:59.734041 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebb1ff-2505-431a-b477-b91b79d032a2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:59.735020 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212de684-17a6-4f07-93b2-a13b11733108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:59.735914 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212de684-17a6-4f07-93b2-a13b11733108 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:59.736196 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212de684-17a6-4f07-93b2-a13b11733108 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:59.736478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212de684-17a6-4f07-93b2-a13b11733108 None admin] PATH_INFO: `/v3/users/cb7c2cbcc6b843458bbdbf6998920f1e` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:59.737804 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212de684-17a6-4f07-93b2-a13b11733108 None admin] RBAC: Authorizing `identity:get_user(user_id=cb7c2cbcc6b843458bbdbf6998920f1e)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:49:59.739596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212de684-17a6-4f07-93b2-a13b11733108 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:49:59.741475 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 97/196] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:49:59 2022] GET /identity/v3/users/cb7c2cbcc6b843458bbdbf6998920f1e => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:49:59.742088 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212de684-17a6-4f07-93b2-a13b11733108 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:59.786834 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228baa0-f073-4801-a1d8-bb7b14ca9572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:59.787700 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228baa0-f073-4801-a1d8-bb7b14ca9572 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:59.788947 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228baa0-f073-4801-a1d8-bb7b14ca9572 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:59.788947 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228baa0-f073-4801-a1d8-bb7b14ca9572 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:59.804840 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228baa0-f073-4801-a1d8-bb7b14ca9572 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:49:59.813213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c228baa0-f073-4801-a1d8-bb7b14ca9572 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:49:59.813213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:49:59.813213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:49:59.813213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:49:59.813213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:49:59.813213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:49:59.813213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:49:59.813213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:49:59.813213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:49:59.813213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:49:59.813213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:49:59.813213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:49:59.813213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:49:59.813213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:49:59.813213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:49:59.813213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:49:59.814646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:49:59.814646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:49:59.814646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:49:59.814646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:49:59.814646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:49:59.814646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:49:59.814646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:49:59.814646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:49:59.814646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:49:59.814646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:49:59.814646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:49:59.814646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:49:59.814646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:49:59.814646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:49:59.814646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:49:59.814646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:49:59.815449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:49:59.815449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:49:59.815449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:49:59.815449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:49:59.815449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:49:59.815449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:49:59.815449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:49:59.815449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:49:59.815449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:49:59.815449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:49:59.815449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:49:59.815449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:49:59.815449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 100/197] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:49:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:49:59.819155 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228baa0-f073-4801-a1d8-bb7b14ca9572 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:49:59.890246 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212de684-17a6-4f07-93b2-a13b11733108 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:59.893978 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:59.935814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:49:59.940431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f6fbc-0c2b-408c-b44c-14db2971e5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:59.941140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f6fbc-0c2b-408c-b44c-14db2971e5de None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:59.941440 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f6fbc-0c2b-408c-b44c-14db2971e5de None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:59.942259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f6fbc-0c2b-408c-b44c-14db2971e5de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:49:59.944282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228baa0-f073-4801-a1d8-bb7b14ca9572 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:49:59.980880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bcb844-43b8-4c07-b600-c2974d4c8abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:49:59.981968 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcb844-43b8-4c07-b600-c2974d4c8abc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:49:59.982484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcb844-43b8-4c07-b600-c2974d4c8abc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:49:59.982962 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcb844-43b8-4c07-b600-c2974d4c8abc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:00.023292 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f6fbc-0c2b-408c-b44c-14db2971e5de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:00.024774 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f6fbc-0c2b-408c-b44c-14db2971e5de None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:00.057413 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bcb844-43b8-4c07-b600-c2974d4c8abc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:00.059280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bcb844-43b8-4c07-b600-c2974d4c8abc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:00.139837 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 98/198] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:49:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:50:00.143564 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f6fbc-0c2b-408c-b44c-14db2971e5de None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:00.147267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:00.167629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:00.180498 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 101/199] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:49:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:50:00.181484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bcb844-43b8-4c07-b600-c2974d4c8abc None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:00.257888 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f6fbc-0c2b-408c-b44c-14db2971e5de None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:00.287454 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bcb844-43b8-4c07-b600-c2974d4c8abc None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:00.295944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a6f32-9a1f-4b0b-9a09-faaed3893269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:00.296910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a6f32-9a1f-4b0b-9a09-faaed3893269 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:00.297435 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a6f32-9a1f-4b0b-9a09-faaed3893269 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:00.297901 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a6f32-9a1f-4b0b-9a09-faaed3893269 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:00.299005 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a6f32-9a1f-4b0b-9a09-faaed3893269 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:00.299507 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a6f32-9a1f-4b0b-9a09-faaed3893269 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:00.301624 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a6f32-9a1f-4b0b-9a09-faaed3893269 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:00.311648 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 99/200] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:50:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:00.313015 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a6f32-9a1f-4b0b-9a09-faaed3893269 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:00.323763 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65021c44-e8d8-4f66-9bac-62fbd04c3c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:00.324634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65021c44-e8d8-4f66-9bac-62fbd04c3c6f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:00.324916 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65021c44-e8d8-4f66-9bac-62fbd04c3c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:00.325190 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65021c44-e8d8-4f66-9bac-62fbd04c3c6f None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:00.326615 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65021c44-e8d8-4f66-9bac-62fbd04c3c6f None admin] RBAC: Authorizing `identity:get_project(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:00.328309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65021c44-e8d8-4f66-9bac-62fbd04c3c6f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:00.329974 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 102/201] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:00 2022] GET /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:00.330788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65021c44-e8d8-4f66-9bac-62fbd04c3c6f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:00.390717 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:00.398936 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:00.430301 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a6f32-9a1f-4b0b-9a09-faaed3893269 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:00.443255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65021c44-e8d8-4f66-9bac-62fbd04c3c6f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:00.472406 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81835dce-8efd-4523-a904-0c644aa62607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:00.474311 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81835dce-8efd-4523-a904-0c644aa62607 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:00.474795 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81835dce-8efd-4523-a904-0c644aa62607 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:00.475312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81835dce-8efd-4523-a904-0c644aa62607 None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:00.476865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81835dce-8efd-4523-a904-0c644aa62607 None admin] RBAC: Authorizing `identity:get_project(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:00.478825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81835dce-8efd-4523-a904-0c644aa62607 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:00.481172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 100/202] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:00 2022] GET /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:00.482264 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81835dce-8efd-4523-a904-0c644aa62607 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:00.488073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0e93d-f9a8-432c-aad3-8640809a9567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:00.489421 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0e93d-f9a8-432c-aad3-8640809a9567 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:00.489704 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0e93d-f9a8-432c-aad3-8640809a9567 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:00.489988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0e93d-f9a8-432c-aad3-8640809a9567 None admin] PATH_INFO: `/v3/users/b087f9732f764f9a9f4c915abe5814c8` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:00.491295 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0e93d-f9a8-432c-aad3-8640809a9567 None admin] RBAC: Authorizing `identity:get_user(user_id=b087f9732f764f9a9f4c915abe5814c8)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:00.492765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0e93d-f9a8-432c-aad3-8640809a9567 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:00.494756 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 103/203] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:50:00 2022] GET /identity/v3/users/b087f9732f764f9a9f4c915abe5814c8 => generated 289 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:00.495615 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0e93d-f9a8-432c-aad3-8640809a9567 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:00.596826 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81835dce-8efd-4523-a904-0c644aa62607 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:00.601670 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0e93d-f9a8-432c-aad3-8640809a9567 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:00.630414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:00.632842 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:00.638350 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63cdd8-dc33-4661-a2c6-a449d5820534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:00.639009 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63cdd8-dc33-4661-a2c6-a449d5820534 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:00.639300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63cdd8-dc33-4661-a2c6-a449d5820534 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:00.639579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63cdd8-dc33-4661-a2c6-a449d5820534 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:00.640197 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd57454-cddd-47a5-94cb-8a9dbebdab75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:00.640413 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63cdd8-dc33-4661-a2c6-a449d5820534 None admin] RBAC: Adding query filter params (role.id=e731efeb1d3c407fb58cf64a02d95796, scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94, user.id=cb7c2cbcc6b843458bbdbf6998920f1e) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:00.640728 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63cdd8-dc33-4661-a2c6-a449d5820534 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:00.641431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd57454-cddd-47a5-94cb-8a9dbebdab75 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:00.641901 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd57454-cddd-47a5-94cb-8a9dbebdab75 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:00.642199 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63cdd8-dc33-4661-a2c6-a449d5820534 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:00.642594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd57454-cddd-47a5-94cb-8a9dbebdab75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:00.643760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd57454-cddd-47a5-94cb-8a9dbebdab75 None admin] RBAC: Adding query filter params (role.id=4fa8f06085c741a5b3cf64e63cc0754f, scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94, user.id=cb7c2cbcc6b843458bbdbf6998920f1e) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:00.644312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd57454-cddd-47a5-94cb-8a9dbebdab75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:00.646066 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd57454-cddd-47a5-94cb-8a9dbebdab75 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:00.648307 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 104/204] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:50:00 2022] GET /identity/v3/role_assignments?user.id=cb7c2cbcc6b843458bbdbf6998920f1e&scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94&role.id=e731efeb1d3c407fb58cf64a02d95796 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:00.652872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 101/205] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:50:00 2022] GET /identity/v3/role_assignments?user.id=cb7c2cbcc6b843458bbdbf6998920f1e&scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94&role.id=4fa8f06085c741a5b3cf64e63cc0754f => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:00.654796 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63cdd8-dc33-4661-a2c6-a449d5820534 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:00.797790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63cdd8-dc33-4661-a2c6-a449d5820534 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:00.852841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea61cf-9816-4b0c-b21a-7f6dd6d77a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:00.854513 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea61cf-9816-4b0c-b21a-7f6dd6d77a24 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:00.855077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea61cf-9816-4b0c-b21a-7f6dd6d77a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:00.855561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea61cf-9816-4b0c-b21a-7f6dd6d77a24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:00.949952 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:00.959275 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea61cf-9816-4b0c-b21a-7f6dd6d77a24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:00.960801 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea61cf-9816-4b0c-b21a-7f6dd6d77a24 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:01.075378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 105/206] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:50:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:50:01.081625 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd57454-cddd-47a5-94cb-8a9dbebdab75 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:01.235373 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd57454-cddd-47a5-94cb-8a9dbebdab75 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:01.291033 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e3114-1038-4c3b-ba82-f4831ac86b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:01.292270 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e3114-1038-4c3b-ba82-f4831ac86b39 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:01.292741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e3114-1038-4c3b-ba82-f4831ac86b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:01.293199 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e3114-1038-4c3b-ba82-f4831ac86b39 None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:01.299218 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e3114-1038-4c3b-ba82-f4831ac86b39 None admin] RBAC: Authorizing `identity:get_project(project_id=89f44d18dc9b464faeb1bf0bd03569af)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:01.301189 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e3114-1038-4c3b-ba82-f4831ac86b39 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:01.303681 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 102/207] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:01 2022] GET /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:01.309956 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea61cf-9816-4b0c-b21a-7f6dd6d77a24 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:01.420633 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:01.428280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea61cf-9816-4b0c-b21a-7f6dd6d77a24 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:01.468975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711fa2e-4bbe-4b4f-bd96-4b71eb732571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:01.469969 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711fa2e-4bbe-4b4f-bd96-4b71eb732571 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:01.470489 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711fa2e-4bbe-4b4f-bd96-4b71eb732571 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:01.470948 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711fa2e-4bbe-4b4f-bd96-4b71eb732571 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:01.472006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711fa2e-4bbe-4b4f-bd96-4b71eb732571 None admin] RBAC: Adding query filter params (role.id=5b3eae4902f44cc9a56bf2ec2a8e13f5, scope.project.id=89f44d18dc9b464faeb1bf0bd03569af, user.id=b087f9732f764f9a9f4c915abe5814c8) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:01.472516 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711fa2e-4bbe-4b4f-bd96-4b71eb732571 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:01.474761 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711fa2e-4bbe-4b4f-bd96-4b71eb732571 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:01.481518 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 106/208] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:50:01 2022] GET /identity/v3/role_assignments?user.id=b087f9732f764f9a9f4c915abe5814c8&scope.project.id=89f44d18dc9b464faeb1bf0bd03569af&role.id=5b3eae4902f44cc9a56bf2ec2a8e13f5 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:02.058958 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2f8ea-ec79-4926-83b3-ee2831700305 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:02.059542 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2f8ea-ec79-4926-83b3-ee2831700305 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:02.060000 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2f8ea-ec79-4926-83b3-ee2831700305 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:02.061595 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 103/209] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:02.067816 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe0f57-c51f-4de6-81e9-4ace8c4381f8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:02.068671 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe0f57-c51f-4de6-81e9-4ace8c4381f8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:02.068959 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe0f57-c51f-4de6-81e9-4ace8c4381f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:02.089305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-85fe0f57-c51f-4de6-81e9-4ace8c4381f8 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:02.090547 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-85fe0f57-c51f-4de6-81e9-4ace8c4381f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:02.188301 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 107/210] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:02.217100 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2f8ea-ec79-4926-83b3-ee2831700305 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:02.300663 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:02.355634 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2f8ea-ec79-4926-83b3-ee2831700305 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:02.363259 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb3118-b6c2-4079-9881-27000eb4d9d4 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:02.364578 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb3118-b6c2-4079-9881-27000eb4d9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:02.365079 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb3118-b6c2-4079-9881-27000eb4d9d4 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:02.367181 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 108/211] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:02.373967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7ff9c-6e43-4b4c-ab75-2558be9e96d7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:02.374336 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7ff9c-6e43-4b4c-ab75-2558be9e96d7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:02.374579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7ff9c-6e43-4b4c-ab75-2558be9e96d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:02.398421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2e94c-4a2a-4363-9b86-691bc182748f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:02.399234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2e94c-4a2a-4363-9b86-691bc182748f None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:02.399503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2e94c-4a2a-4363-9b86-691bc182748f None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:02.399787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2e94c-4a2a-4363-9b86-691bc182748f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:02.400783 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8af7ff9c-6e43-4b4c-ab75-2558be9e96d7 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:02.403131 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8af7ff9c-6e43-4b4c-ab75-2558be9e96d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:02.412262 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2e94c-4a2a-4363-9b86-691bc182748f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:02.421872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-58d2e94c-4a2a-4363-9b86-691bc182748f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:02.421872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:02.421872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:02.421872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:02.421872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:02.421872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:02.421872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:02.421872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:02.421872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:02.421872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:02.421872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:02.421872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:02.421872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:02.421872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:02.421872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:02.421872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:02.422885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:02.422885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:02.422885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:02.422885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:02.422885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:02.422885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:02.422885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:02.422885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:02.422885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:02.422885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:02.422885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:02.422885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:02.422885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:02.422885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:02.422885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:02.422885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:02.427368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:02.427368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:02.427368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:02.427368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:02.427368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:02.427368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:02.427368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:02.427368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:02.427368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:02.427368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:02.427368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:02.427368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:02.427368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 104/212] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:02.429629 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2e94c-4a2a-4363-9b86-691bc182748f None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:02.482592 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 109/213] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:50:02.508544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7ff9c-6e43-4b4c-ab75-2558be9e96d7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:02.534818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:02.558999 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2e94c-4a2a-4363-9b86-691bc182748f None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:02.584771 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:02.599928 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44e6f1-1567-4ab1-9ec3-35e6d62875fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:02.600955 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44e6f1-1567-4ab1-9ec3-35e6d62875fe None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:02.601454 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44e6f1-1567-4ab1-9ec3-35e6d62875fe None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:02.601948 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44e6f1-1567-4ab1-9ec3-35e6d62875fe None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:02.603043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44e6f1-1567-4ab1-9ec3-35e6d62875fe None admin] RBAC: Adding query filter params (name=member) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:02.603548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44e6f1-1567-4ab1-9ec3-35e6d62875fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:02.605118 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44e6f1-1567-4ab1-9ec3-35e6d62875fe None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:02.615403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 105/214] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:50:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:02.621588 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44e6f1-1567-4ab1-9ec3-35e6d62875fe None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:02.640097 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7ff9c-6e43-4b4c-ab75-2558be9e96d7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:02.678082 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173978ea-dd21-43dc-9f0b-71ec75052490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:02.679191 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173978ea-dd21-43dc-9f0b-71ec75052490 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:02.679649 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173978ea-dd21-43dc-9f0b-71ec75052490 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:02.680087 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173978ea-dd21-43dc-9f0b-71ec75052490 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:02.681430 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173978ea-dd21-43dc-9f0b-71ec75052490 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:02.683128 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173978ea-dd21-43dc-9f0b-71ec75052490 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:02.684959 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 110/215] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:02.691027 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173978ea-dd21-43dc-9f0b-71ec75052490 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:02.747318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44e6f1-1567-4ab1-9ec3-35e6d62875fe None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:02.788826 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56711c41-7570-4244-853d-0fc4323532d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:02.790441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56711c41-7570-4244-853d-0fc4323532d5 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:02.790941 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56711c41-7570-4244-853d-0fc4323532d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:02.791531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56711c41-7570-4244-853d-0fc4323532d5 None admin] PATH_INFO: `/v3/users/4d6bf94fec774561b130aa7ebf72152b` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:02.795939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:02.809859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173978ea-dd21-43dc-9f0b-71ec75052490 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:02.811443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56711c41-7570-4244-853d-0fc4323532d5 None admin] RBAC: Authorizing `identity:get_user(user_id=4d6bf94fec774561b130aa7ebf72152b)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:02.813236 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56711c41-7570-4244-853d-0fc4323532d5 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:02.815503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 106/216] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:50:02 2022] GET /identity/v3/users/4d6bf94fec774561b130aa7ebf72152b => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:02.823815 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56711c41-7570-4244-853d-0fc4323532d5 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:02.858988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f0e19-b917-451d-bede-188a45880f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:02.860106 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f0e19-b917-451d-bede-188a45880f88 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:02.860393 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f0e19-b917-451d-bede-188a45880f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:02.860667 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f0e19-b917-451d-bede-188a45880f88 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:02.861652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f0e19-b917-451d-bede-188a45880f88 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:02.864073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f0e19-b917-451d-bede-188a45880f88 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:02.886904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:02.910994 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 111/217] 10.222.0.76 () {60 vars in 1132 bytes} [Thu May 26 17:50:02 2022] POST /identity/v3/users => generated 297 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:50:02.967156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56711c41-7570-4244-853d-0fc4323532d5 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:03.009942 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c0857-b4c0-499c-9e31-6b4a648a1c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:03.011067 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c0857-b4c0-499c-9e31-6b4a648a1c14 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:03.011584 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c0857-b4c0-499c-9e31-6b4a648a1c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:03.012109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c0857-b4c0-499c-9e31-6b4a648a1c14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:03.103050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c0857-b4c0-499c-9e31-6b4a648a1c14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:03.104548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c0857-b4c0-499c-9e31-6b4a648a1c14 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:03.159662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:03.224141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 107/218] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:50:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 17:50:03.230480 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f0e19-b917-451d-bede-188a45880f88 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:03.357947 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f0e19-b917-451d-bede-188a45880f88 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:03.395340 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627e3d6-4a70-4201-b140-6f6557fd3602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:03.396662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627e3d6-4a70-4201-b140-6f6557fd3602 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:03.396999 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627e3d6-4a70-4201-b140-6f6557fd3602 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:03.397219 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627e3d6-4a70-4201-b140-6f6557fd3602 None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:03.398549 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627e3d6-4a70-4201-b140-6f6557fd3602 None admin] RBAC: Authorizing `identity:get_project(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:03.400032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627e3d6-4a70-4201-b140-6f6557fd3602 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:03.401769 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 112/219] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:03 2022] GET /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:03.407662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c0857-b4c0-499c-9e31-6b4a648a1c14 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:03.520414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532271a-7a52-4e82-9c97-e7726e09e31f None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:03.521174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532271a-7a52-4e82-9c97-e7726e09e31f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:03.521680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532271a-7a52-4e82-9c97-e7726e09e31f None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:03.523782 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 113/220] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:03.529308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a620da4-2f67-4629-8744-9a4a879002e8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:03.529610 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a620da4-2f67-4629-8744-9a4a879002e8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:03.529877 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a620da4-2f67-4629-8744-9a4a879002e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:03.539900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c0857-b4c0-499c-9e31-6b4a648a1c14 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:03.552380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-9a620da4-2f67-4629-8744-9a4a879002e8 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:03.553783 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9a620da4-2f67-4629-8744-9a4a879002e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:03.563004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:03.582639 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0e86d-c92f-4864-bc94-eb50f20dd09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:03.583829 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0e86d-c92f-4864-bc94-eb50f20dd09b None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:03.584419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0e86d-c92f-4864-bc94-eb50f20dd09b None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:03.584712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0e86d-c92f-4864-bc94-eb50f20dd09b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:03.585571 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0e86d-c92f-4864-bc94-eb50f20dd09b None admin] RBAC: Adding query filter params (role.id=eb1858a5621543028836223abcb3c0a1, scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94, user.id=4d6bf94fec774561b130aa7ebf72152b) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:03.586062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0e86d-c92f-4864-bc94-eb50f20dd09b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:03.588334 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0e86d-c92f-4864-bc94-eb50f20dd09b None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:03.597933 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 108/221] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:50:03 2022] GET /identity/v3/role_assignments?user.id=4d6bf94fec774561b130aa7ebf72152b&scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94&role.id=eb1858a5621543028836223abcb3c0a1 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:03.599738 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808af303-a038-47dd-b243-3d2d0da7934d None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:03.600044 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808af303-a038-47dd-b243-3d2d0da7934d None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:03.601298 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808af303-a038-47dd-b243-3d2d0da7934d None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:03.603079 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 109/222] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:03.610420 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42941cd1-187d-4dbb-9b9d-9070d007c1b6 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:03.610744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42941cd1-187d-4dbb-9b9d-9070d007c1b6 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:03.611029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42941cd1-187d-4dbb-9b9d-9070d007c1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:03.633138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-42941cd1-187d-4dbb-9b9d-9070d007c1b6 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:03.634412 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-42941cd1-187d-4dbb-9b9d-9070d007c1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:03.653789 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 114/223] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:03.655648 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:03.688025 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a620da4-2f67-4629-8744-9a4a879002e8 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:03.722272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 110/224] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:50:03.767368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42941cd1-187d-4dbb-9b9d-9070d007c1b6 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:03.889045 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a620da4-2f67-4629-8744-9a4a879002e8 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:03.931043 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628450f0-294e-4892-bfb6-6f2536864e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:03.932081 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628450f0-294e-4892-bfb6-6f2536864e4f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:03.932590 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628450f0-294e-4892-bfb6-6f2536864e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:03.933007 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628450f0-294e-4892-bfb6-6f2536864e4f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:03.946558 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628450f0-294e-4892-bfb6-6f2536864e4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:03.951076 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42941cd1-187d-4dbb-9b9d-9070d007c1b6 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:03.956426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-628450f0-294e-4892-bfb6-6f2536864e4f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:50:03.956426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:03.956426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:03.956426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:03.956426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:03.956426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:03.956426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:03.956426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:03.956426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:03.956426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:03.956426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:03.956426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:03.956426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:03.956426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:03.956426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:03.956426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:03.957447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:03.957447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:03.957447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:03.957447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:03.957447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:03.957447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:03.957447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:03.957447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:03.957447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:03.957447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:03.957447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:03.957447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:03.957447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:03.957447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:03.957447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:03.957447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:03.958368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:03.958368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:03.958368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:03.958368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:03.958368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:03.958368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:03.958368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:03.958368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:03.958368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:03.958368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:03.958368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:50:03.958368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:03.961222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 115/225] 10.222.0.76 () {56 vars in 1114 bytes} [Thu May 26 17:50:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:03.966580 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628450f0-294e-4892-bfb6-6f2536864e4f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:04.006553 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:04.021948 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eaa0de-6763-4fe1-b1fd-d4a06b26e2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:04.022804 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eaa0de-6763-4fe1-b1fd-d4a06b26e2dc None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:04.023096 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eaa0de-6763-4fe1-b1fd-d4a06b26e2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:04.023435 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eaa0de-6763-4fe1-b1fd-d4a06b26e2dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:04.040026 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:04.050237 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eaa0de-6763-4fe1-b1fd-d4a06b26e2dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:04.067515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-71eaa0de-6763-4fe1-b1fd-d4a06b26e2dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:50:04.067515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:04.067515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:04.067515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:04.067515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:04.067515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:04.067515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:04.067515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:04.067515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:04.067515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:04.067515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:04.067515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:04.067515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:04.067515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:04.067515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:04.067515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:04.068486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:04.068486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:04.068486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:04.068486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:04.068486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:04.068486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:04.068486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:04.068486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:04.068486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:04.068486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:04.068486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:04.068486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:04.068486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:04.068486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:04.068486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:04.068486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:04.069272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:04.069272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:04.069272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:04.069272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:04.069272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:04.069272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:04.069272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:04.069272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:04.069272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:04.069272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:04.069272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:50:04.069272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:04.069272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 111/226] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:50:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:04.087291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eaa0de-6763-4fe1-b1fd-d4a06b26e2dc None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:04.087921 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628450f0-294e-4892-bfb6-6f2536864e4f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:04.135570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b943e-ee66-484a-a7c3-f504402f9482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:04.136381 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b943e-ee66-484a-a7c3-f504402f9482 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:04.136758 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b943e-ee66-484a-a7c3-f504402f9482 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:04.137107 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b943e-ee66-484a-a7c3-f504402f9482 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:04.143123 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9b943e-ee66-484a-a7c3-f504402f9482 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:04.143502 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9b943e-ee66-484a-a7c3-f504402f9482 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:04.145567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9b943e-ee66-484a-a7c3-f504402f9482 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:04.156293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 116/227] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:50:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:04.162228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b943e-ee66-484a-a7c3-f504402f9482 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:04.260988 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eaa0de-6763-4fe1-b1fd-d4a06b26e2dc None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:04.303500 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907442d-24a5-4a8f-bb49-f770d9275737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:04.304232 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907442d-24a5-4a8f-bb49-f770d9275737 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:04.304519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907442d-24a5-4a8f-bb49-f770d9275737 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:04.304800 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907442d-24a5-4a8f-bb49-f770d9275737 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:04.305570 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907442d-24a5-4a8f-bb49-f770d9275737 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:04.305884 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907442d-24a5-4a8f-bb49-f770d9275737 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:04.308146 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907442d-24a5-4a8f-bb49-f770d9275737 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:04.324253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 112/228] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:04.325630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907442d-24a5-4a8f-bb49-f770d9275737 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:04.394638 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:04.420614 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:04.477013 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907442d-24a5-4a8f-bb49-f770d9275737 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:04.508278 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b943e-ee66-484a-a7c3-f504402f9482 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:04.559646 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230aca22-9100-45a0-a1e1-399d6b45c54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:04.560477 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230aca22-9100-45a0-a1e1-399d6b45c54b None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:04.560761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230aca22-9100-45a0-a1e1-399d6b45c54b None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:04.561036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230aca22-9100-45a0-a1e1-399d6b45c54b None admin] PATH_INFO: `/v3/users/cb7c2cbcc6b843458bbdbf6998920f1e` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:04.563215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230aca22-9100-45a0-a1e1-399d6b45c54b None admin] RBAC: Authorizing `identity:get_user(user_id=cb7c2cbcc6b843458bbdbf6998920f1e)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:04.564803 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230aca22-9100-45a0-a1e1-399d6b45c54b None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:04.567266 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 113/229] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:50:04 2022] GET /identity/v3/users/cb7c2cbcc6b843458bbdbf6998920f1e => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:04.572235 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6f8fc-ac4f-42d0-8a78-839d54cdb4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:04.573151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6f8fc-ac4f-42d0-8a78-839d54cdb4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:04.573431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6f8fc-ac4f-42d0-8a78-839d54cdb4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:04.573707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6f8fc-ac4f-42d0-8a78-839d54cdb4d9 None admin] PATH_INFO: `/v3/users/cb7c2cbcc6b843458bbdbf6998920f1e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:04.575128 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6f8fc-ac4f-42d0-8a78-839d54cdb4d9 None admin] RBAC: Authorizing `identity:get_user(user_id=cb7c2cbcc6b843458bbdbf6998920f1e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:04.577001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230aca22-9100-45a0-a1e1-399d6b45c54b None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:04.578425 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6f8fc-ac4f-42d0-8a78-839d54cdb4d9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:04.583954 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 117/230] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:50:04 2022] GET /identity/v3/users/cb7c2cbcc6b843458bbdbf6998920f1e => generated 283 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:04.590348 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6f8fc-ac4f-42d0-8a78-839d54cdb4d9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:04.783268 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230aca22-9100-45a0-a1e1-399d6b45c54b None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:04.798972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6f8fc-ac4f-42d0-8a78-839d54cdb4d9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:04.836142 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:04.851122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43099ac-8607-47e5-8fd3-0b784492121a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:04.851849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:04.852024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43099ac-8607-47e5-8fd3-0b784492121a None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:04.852315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43099ac-8607-47e5-8fd3-0b784492121a None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:04.852591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43099ac-8607-47e5-8fd3-0b784492121a None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:04.854257 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43099ac-8607-47e5-8fd3-0b784492121a None admin] RBAC: Authorizing `identity:get_project(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:04.856486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43099ac-8607-47e5-8fd3-0b784492121a None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:04.859508 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 114/231] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:04 2022] GET /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94 => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:04.861967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74784ed2-7341-42ac-b638-2b3b129ade75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:04.862890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74784ed2-7341-42ac-b638-2b3b129ade75 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:04.863169 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74784ed2-7341-42ac-b638-2b3b129ade75 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:04.863984 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74784ed2-7341-42ac-b638-2b3b129ade75 None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:04.865235 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74784ed2-7341-42ac-b638-2b3b129ade75 None admin] RBAC: Authorizing `identity:get_project(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:04.867455 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43099ac-8607-47e5-8fd3-0b784492121a None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:04.868617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74784ed2-7341-42ac-b638-2b3b129ade75 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:04.873514 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 118/232] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:04 2022] GET /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94 => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:04.879662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74784ed2-7341-42ac-b638-2b3b129ade75 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:05.063784 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43099ac-8607-47e5-8fd3-0b784492121a None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:05.100945 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74784ed2-7341-42ac-b638-2b3b129ade75 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:05.104871 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63a7d4-755b-49d7-9206-5af2edcc67c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:05.105785 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63a7d4-755b-49d7-9206-5af2edcc67c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:05.106069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63a7d4-755b-49d7-9206-5af2edcc67c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:05.106496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63a7d4-755b-49d7-9206-5af2edcc67c4 None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94/users/cb7c2cbcc6b843458bbdbf6998920f1e/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:05.110814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63a7d4-755b-49d7-9206-5af2edcc67c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94, user_id=cb7c2cbcc6b843458bbdbf6998920f1e, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:05.112289 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63a7d4-755b-49d7-9206-5af2edcc67c4 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:05.137277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 115/233] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:50:04 2022] PUT /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94/users/cb7c2cbcc6b843458bbdbf6998920f1e/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 17:50:05.199127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fc0de-03d1-43cf-8f47-3fbaea3416b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:05.200074 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fc0de-03d1-43cf-8f47-3fbaea3416b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:05.200368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fc0de-03d1-43cf-8f47-3fbaea3416b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:05.200650 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fc0de-03d1-43cf-8f47-3fbaea3416b5 None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94/users/cb7c2cbcc6b843458bbdbf6998920f1e/roles/e731efeb1d3c407fb58cf64a02d95796` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:05.203221 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fc0de-03d1-43cf-8f47-3fbaea3416b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94, user_id=cb7c2cbcc6b843458bbdbf6998920f1e, role_id=e731efeb1d3c407fb58cf64a02d95796)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:05.204724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fc0de-03d1-43cf-8f47-3fbaea3416b5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:05.225320 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 119/234] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:50:04 2022] PUT /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94/users/cb7c2cbcc6b843458bbdbf6998920f1e/roles/e731efeb1d3c407fb58cf64a02d95796 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:50:05.867825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfca41-19d1-422f-b458-80e45c4f4d03 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:05.868931 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfca41-19d1-422f-b458-80e45c4f4d03 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:05.869427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfca41-19d1-422f-b458-80e45c4f4d03 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:05.871415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 116/235] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:05.880785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff22d96-ed29-447d-b893-d96b45304657 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:05.881285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff22d96-ed29-447d-b893-d96b45304657 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:05.881359 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff22d96-ed29-447d-b893-d96b45304657 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:05.882817 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 120/236] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:05.884568 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14efd5-4bc6-469e-b503-68bb950675d5 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:05.884855 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14efd5-4bc6-469e-b503-68bb950675d5 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:05.885136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14efd5-4bc6-469e-b503-68bb950675d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:05.892030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d14374-585d-4c64-9509-0483ad4899d5 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:05.892312 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d14374-585d-4c64-9509-0483ad4899d5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:05.892591 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d14374-585d-4c64-9509-0483ad4899d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:05.918265 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-ea14efd5-4bc6-469e-b503-68bb950675d5 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:05.919500 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-ea14efd5-4bc6-469e-b503-68bb950675d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:05.927680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a7d14374-585d-4c64-9509-0483ad4899d5 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:05.928884 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a7d14374-585d-4c64-9509-0483ad4899d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:06.015920 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:06.041953 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:06.044926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 117/237] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:06.072486 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 121/238] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:06.083547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14efd5-4bc6-469e-b503-68bb950675d5 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:06.107637 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d14374-585d-4c64-9509-0483ad4899d5 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:06.281685 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d14374-585d-4c64-9509-0483ad4899d5 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:06.285851 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14efd5-4bc6-469e-b503-68bb950675d5 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:06.333522 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e0d30-c7b9-4657-b741-326d92109e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:06.334397 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e0d30-c7b9-4657-b741-326d92109e24 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:06.334676 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e0d30-c7b9-4657-b741-326d92109e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:06.334958 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e0d30-c7b9-4657-b741-326d92109e24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:06.336155 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e0d30-c7b9-4657-b741-326d92109e24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:06.338021 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e0d30-c7b9-4657-b741-326d92109e24 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:06.339877 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 118/239] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:06.340584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8039a-108a-4d45-9ef3-40e208edbb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:06.341399 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8039a-108a-4d45-9ef3-40e208edbb85 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:06.341746 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8039a-108a-4d45-9ef3-40e208edbb85 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:06.342030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8039a-108a-4d45-9ef3-40e208edbb85 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:06.345369 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e0d30-c7b9-4657-b741-326d92109e24 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:06.369590 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8039a-108a-4d45-9ef3-40e208edbb85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:06.386246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-7ff8039a-108a-4d45-9ef3-40e208edbb85 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:50:06.386246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:06.386246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:06.386246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:06.386246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:06.386246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:06.386246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:06.386246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:06.386246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:06.386246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:06.386246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:06.386246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:06.386246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:06.386246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:06.386246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:06.386246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:06.388661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:06.388661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:06.388661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:06.388661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:06.388661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:06.388661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:06.388661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:06.388661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:06.388661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:06.388661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:06.388661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:06.388661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:06.388661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:06.388661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:06.388661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:06.388661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:06.390356 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:06.390356 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:06.390356 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:06.390356 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:06.390356 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:06.390356 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:06.390356 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:06.390356 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:06.390356 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:06.390356 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:06.390356 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:50:06.390356 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:06.390356 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 122/240] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:50:06.398317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8039a-108a-4d45-9ef3-40e208edbb85 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:06.418565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:06.505277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e0d30-c7b9-4657-b741-326d92109e24 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:06.507716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:06.548340 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974f65e-04b2-4a9a-bc41-f49700cb89f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:06.549272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974f65e-04b2-4a9a-bc41-f49700cb89f7 None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:06.549736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974f65e-04b2-4a9a-bc41-f49700cb89f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:06.550165 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974f65e-04b2-4a9a-bc41-f49700cb89f7 None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:06.551195 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974f65e-04b2-4a9a-bc41-f49700cb89f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:06.552685 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974f65e-04b2-4a9a-bc41-f49700cb89f7 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:06.571209 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8039a-108a-4d45-9ef3-40e208edbb85 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:06.584001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 119/241] 10.222.0.76 () {60 vars in 1132 bytes} [Thu May 26 17:50:06 2022] POST /identity/v3/users => generated 293 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:50:06.585953 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79f723-d364-4a61-817e-c9331d756011 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:06.586481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79f723-d364-4a61-817e-c9331d756011 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:06.587021 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79f723-d364-4a61-817e-c9331d756011 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:06.588736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 120/242] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:06.596310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcb363-ec88-491a-a87d-421c22a0ba8e None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:06.596310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcb363-ec88-491a-a87d-421c22a0ba8e None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:06.596310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcb363-ec88-491a-a87d-421c22a0ba8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:06.629469 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-44bcb363-ec88-491a-a87d-421c22a0ba8e None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:06.632254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-44bcb363-ec88-491a-a87d-421c22a0ba8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:06.649986 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f46d8a-9769-4972-ade5-256440629a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:06.650691 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f46d8a-9769-4972-ade5-256440629a11 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:06.651039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f46d8a-9769-4972-ade5-256440629a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:06.651322 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f46d8a-9769-4972-ade5-256440629a11 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:06.652162 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f46d8a-9769-4972-ade5-256440629a11 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:06.652480 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f46d8a-9769-4972-ade5-256440629a11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:06.653930 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f46d8a-9769-4972-ade5-256440629a11 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:06.674728 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 123/243] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:50:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:06.681092 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f46d8a-9769-4972-ade5-256440629a11 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:06.800116 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 121/244] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:06.826836 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bcb363-ec88-491a-a87d-421c22a0ba8e None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:06.836264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f46d8a-9769-4972-ade5-256440629a11 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:06.858766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:06.890137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22362ee9-1d10-4a99-8070-798b1c5fac4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:06.891202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22362ee9-1d10-4a99-8070-798b1c5fac4b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:06.891481 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22362ee9-1d10-4a99-8070-798b1c5fac4b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:06.891750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22362ee9-1d10-4a99-8070-798b1c5fac4b None admin] PATH_INFO: `/v3/users/c6e310df95684295b07dc76b8cc0c4b4` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:06.897068 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:06.919846 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22362ee9-1d10-4a99-8070-798b1c5fac4b None admin] RBAC: Authorizing `identity:get_user(user_id=c6e310df95684295b07dc76b8cc0c4b4)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:06.921812 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22362ee9-1d10-4a99-8070-798b1c5fac4b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:06.924390 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 124/245] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:50:06 2022] GET /identity/v3/users/c6e310df95684295b07dc76b8cc0c4b4 => generated 297 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:06.930030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22362ee9-1d10-4a99-8070-798b1c5fac4b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:06.983017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bcb363-ec88-491a-a87d-421c22a0ba8e None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:07.061316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d15f1-3ad5-4d27-9d6e-927c5a1e4166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:07.062545 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d15f1-3ad5-4d27-9d6e-927c5a1e4166 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:07.063032 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d15f1-3ad5-4d27-9d6e-927c5a1e4166 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:07.063645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d15f1-3ad5-4d27-9d6e-927c5a1e4166 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:07.083812 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22362ee9-1d10-4a99-8070-798b1c5fac4b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:07.085768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d15f1-3ad5-4d27-9d6e-927c5a1e4166 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:07.094719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-126d15f1-3ad5-4d27-9d6e-927c5a1e4166 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:07.094719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:07.094719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:07.094719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:07.094719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:07.094719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:07.094719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:07.094719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:07.094719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:07.094719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:07.094719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:07.094719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:07.094719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:07.094719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:07.094719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:07.094719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:07.095695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:07.095695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:07.095695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:07.095695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:07.095695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:07.095695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:07.095695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:07.095695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:07.095695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:07.095695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:07.095695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:07.095695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:07.095695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:07.095695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:07.095695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:07.095695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:07.096780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:07.096780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:07.096780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:07.096780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:07.096780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:07.096780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:07.096780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:07.096780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:07.096780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:07.096780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:07.096780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:07.096780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:07.096780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 122/246] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:50:07.104721 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d15f1-3ad5-4d27-9d6e-927c5a1e4166 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:07.140621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd9b83-04c6-48e6-b922-ead11931cc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:07.146028 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd9b83-04c6-48e6-b922-ead11931cc83 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:07.146621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd9b83-04c6-48e6-b922-ead11931cc83 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:07.147125 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd9b83-04c6-48e6-b922-ead11931cc83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:07.201627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:07.234111 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd9b83-04c6-48e6-b922-ead11931cc83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:07.235888 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd9b83-04c6-48e6-b922-ead11931cc83 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:07.242448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:07.246614 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d15f1-3ad5-4d27-9d6e-927c5a1e4166 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:07.300317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e675e-3ca7-4d91-989e-d08b9d6f5000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:07.301032 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e675e-3ca7-4d91-989e-d08b9d6f5000 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:07.301720 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e675e-3ca7-4d91-989e-d08b9d6f5000 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:07.301720 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e675e-3ca7-4d91-989e-d08b9d6f5000 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:07.302495 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265e675e-3ca7-4d91-989e-d08b9d6f5000 None admin] RBAC: Adding query filter params (name=member) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:07.302877 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265e675e-3ca7-4d91-989e-d08b9d6f5000 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:07.304315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265e675e-3ca7-4d91-989e-d08b9d6f5000 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:07.318138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 123/247] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:50:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:07.323516 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e675e-3ca7-4d91-989e-d08b9d6f5000 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:07.373884 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 125/248] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:50:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:50:07.383678 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd9b83-04c6-48e6-b922-ead11931cc83 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:07.503336 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e675e-3ca7-4d91-989e-d08b9d6f5000 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:07.535963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:07.546318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4b970-0ab6-4449-89ee-227651468bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:07.547613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4b970-0ab6-4449-89ee-227651468bdd None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:07.547613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4b970-0ab6-4449-89ee-227651468bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:07.547780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4b970-0ab6-4449-89ee-227651468bdd None admin] PATH_INFO: `/v3/users/4d6bf94fec774561b130aa7ebf72152b` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:07.549063 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4b970-0ab6-4449-89ee-227651468bdd None admin] RBAC: Authorizing `identity:get_user(user_id=4d6bf94fec774561b130aa7ebf72152b)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:07.550613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4b970-0ab6-4449-89ee-227651468bdd None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:07.553194 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 124/249] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:50:07 2022] GET /identity/v3/users/4d6bf94fec774561b130aa7ebf72152b => generated 297 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:07.559104 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4b970-0ab6-4449-89ee-227651468bdd None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:07.585231 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd9b83-04c6-48e6-b922-ead11931cc83 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:07.610715 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:07.624444 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd790c-a218-4b5f-b38c-bc17862407ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:07.625376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd790c-a218-4b5f-b38c-bc17862407ba None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:07.626024 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd790c-a218-4b5f-b38c-bc17862407ba None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:07.626376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd790c-a218-4b5f-b38c-bc17862407ba None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:07.627698 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd790c-a218-4b5f-b38c-bc17862407ba None admin] RBAC: Authorizing `identity:get_project(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:07.629554 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd790c-a218-4b5f-b38c-bc17862407ba None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:07.631798 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 126/250] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:07 2022] GET /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:07.639092 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd790c-a218-4b5f-b38c-bc17862407ba None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:07.770871 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4b970-0ab6-4449-89ee-227651468bdd None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:07.814979 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802ca97-f05d-4342-a8dd-cfd0d97335ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:07.815875 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802ca97-f05d-4342-a8dd-cfd0d97335ec None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:07.816163 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802ca97-f05d-4342-a8dd-cfd0d97335ec None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:07.817244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802ca97-f05d-4342-a8dd-cfd0d97335ec None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:07.818254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9802ca97-f05d-4342-a8dd-cfd0d97335ec None admin] RBAC: Authorizing `identity:get_project(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:07.819943 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9802ca97-f05d-4342-a8dd-cfd0d97335ec None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:07.822278 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 125/251] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:07 2022] GET /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94 => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:07.828779 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802ca97-f05d-4342-a8dd-cfd0d97335ec None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:07.947999 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802ca97-f05d-4342-a8dd-cfd0d97335ec None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:07.958638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:07.980110 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd790c-a218-4b5f-b38c-bc17862407ba None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:07.984955 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390d4b0-4ba8-4a2d-bd0c-848d61760e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:07.985987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390d4b0-4ba8-4a2d-bd0c-848d61760e68 None admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:07.986428 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390d4b0-4ba8-4a2d-bd0c-848d61760e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:07.986821 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390d4b0-4ba8-4a2d-bd0c-848d61760e68 None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94/users/4d6bf94fec774561b130aa7ebf72152b/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:07.989272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9390d4b0-4ba8-4a2d-bd0c-848d61760e68 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94, user_id=4d6bf94fec774561b130aa7ebf72152b, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:07.990977 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9390d4b0-4ba8-4a2d-bd0c-848d61760e68 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:08.008126 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 126/252] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:50:07 2022] PUT /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94/users/4d6bf94fec774561b130aa7ebf72152b/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 17:50:08.061915 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ea8ce-c8fe-4b3e-accc-91a72c469ea9 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:08.062231 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ea8ce-c8fe-4b3e-accc-91a72c469ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:08.062450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ea8ce-c8fe-4b3e-accc-91a72c469ea9 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:08.063860 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 127/253] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:50:08.066225 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248456c-ec6f-487f-b0f4-a72a173829bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:08.068069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248456c-ec6f-487f-b0f4-a72a173829bb None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:08.068558 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248456c-ec6f-487f-b0f4-a72a173829bb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:08.069049 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248456c-ec6f-487f-b0f4-a72a173829bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:08.070101 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248456c-ec6f-487f-b0f4-a72a173829bb None admin] RBAC: Adding query filter params (role.id=5b3eae4902f44cc9a56bf2ec2a8e13f5, scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94, user.id=c6e310df95684295b07dc76b8cc0c4b4) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:08.070697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248456c-ec6f-487f-b0f4-a72a173829bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:08.072444 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248456c-ec6f-487f-b0f4-a72a173829bb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:08.077049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c159a2b-abd9-4926-b4d1-733f20462228 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:08.077334 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c159a2b-abd9-4926-b4d1-733f20462228 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:08.077616 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c159a2b-abd9-4926-b4d1-733f20462228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:08.080374 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 127/254] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:50:07 2022] GET /identity/v3/role_assignments?user.id=c6e310df95684295b07dc76b8cc0c4b4&scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94&role.id=5b3eae4902f44cc9a56bf2ec2a8e13f5 => generated 259 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:08.111438 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-6c159a2b-abd9-4926-b4d1-733f20462228 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:08.112706 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-6c159a2b-abd9-4926-b4d1-733f20462228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:08.237089 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 128/255] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:08.275797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248456c-ec6f-487f-b0f4-a72a173829bb None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:08.278960 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:08.435952 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248456c-ec6f-487f-b0f4-a72a173829bb None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:08.503910 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abba402-1463-43f7-82fd-6240e845690e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:08.505043 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abba402-1463-43f7-82fd-6240e845690e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:08.505331 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abba402-1463-43f7-82fd-6240e845690e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:08.507452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abba402-1463-43f7-82fd-6240e845690e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:08.524675 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abba402-1463-43f7-82fd-6240e845690e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:08.537384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-6abba402-1463-43f7-82fd-6240e845690e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:50:08.537384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:08.537384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:08.537384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:08.537384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:08.537384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:08.537384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:08.537384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:08.537384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:08.537384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:08.537384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:08.537384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:08.537384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:08.537384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:08.537384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:08.537384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:08.538377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:08.538377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:08.538377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:08.538377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:08.538377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:08.538377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:08.538377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:08.538377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:08.538377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:08.538377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:08.538377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:08.538377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:08.538377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:08.538377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:08.538377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:08.538377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:08.539198 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:08.539198 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:08.539198 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:08.539198 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:08.539198 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:08.539198 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:08.539198 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:08.539198 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:08.539198 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:08.539198 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:08.539198 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:50:08.539198 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:08.542297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 128/256] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:50:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:08.547364 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c159a2b-abd9-4926-b4d1-733f20462228 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:08.718724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:08.729925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c159a2b-abd9-4926-b4d1-733f20462228 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:08.753983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d41ac-53f9-45a4-b632-54f92ada3fbf None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:08.758741 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d41ac-53f9-45a4-b632-54f92ada3fbf None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:08.759331 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d41ac-53f9-45a4-b632-54f92ada3fbf None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:08.761429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 129/257] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:08.766600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362b886-75ba-4416-a6d8-6afdfa0a2cb2 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:08.767256 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362b886-75ba-4416-a6d8-6afdfa0a2cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:08.767826 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362b886-75ba-4416-a6d8-6afdfa0a2cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:08.782224 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500bf4f2-01d2-4d09-9002-550f5b116b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:08.782224 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bf4f2-01d2-4d09-9002-550f5b116b3c None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:08.782224 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bf4f2-01d2-4d09-9002-550f5b116b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:08.782224 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bf4f2-01d2-4d09-9002-550f5b116b3c None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:08.782649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500bf4f2-01d2-4d09-9002-550f5b116b3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:08.782649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500bf4f2-01d2-4d09-9002-550f5b116b3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:08.782977 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500bf4f2-01d2-4d09-9002-550f5b116b3c None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:08.798223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 129/258] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:08.802447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d362b886-75ba-4416-a6d8-6afdfa0a2cb2 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:08.804076 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d362b886-75ba-4416-a6d8-6afdfa0a2cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:08.812093 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500bf4f2-01d2-4d09-9002-550f5b116b3c None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:08.883630 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:08.913751 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 130/259] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:08.970017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500bf4f2-01d2-4d09-9002-550f5b116b3c None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:08.990709 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362b886-75ba-4416-a6d8-6afdfa0a2cb2 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:09.021430 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fca89f-971b-42d5-9924-ff215a759850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:09.021430 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fca89f-971b-42d5-9924-ff215a759850 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:09.021430 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fca89f-971b-42d5-9924-ff215a759850 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:09.021430 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fca89f-971b-42d5-9924-ff215a759850 None admin] PATH_INFO: `/v3/users/cb7c2cbcc6b843458bbdbf6998920f1e` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:09.022719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fca89f-971b-42d5-9924-ff215a759850 None admin] RBAC: Authorizing `identity:get_user(user_id=cb7c2cbcc6b843458bbdbf6998920f1e)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:09.024909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fca89f-971b-42d5-9924-ff215a759850 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:09.027403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 130/260] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:50:08 2022] GET /identity/v3/users/cb7c2cbcc6b843458bbdbf6998920f1e => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:09.036118 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fca89f-971b-42d5-9924-ff215a759850 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:09.067215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:09.210228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fca89f-971b-42d5-9924-ff215a759850 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:09.287213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8a56c-0c7d-47f5-91e8-ac56d58f5a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:09.287883 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8a56c-0c7d-47f5-91e8-ac56d58f5a2f None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:09.288167 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8a56c-0c7d-47f5-91e8-ac56d58f5a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:09.288441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8a56c-0c7d-47f5-91e8-ac56d58f5a2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:09.291263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362b886-75ba-4416-a6d8-6afdfa0a2cb2 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:09.353553 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf71c2-0f94-4bc0-bf55-497402c1809c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:09.354405 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf71c2-0f94-4bc0-bf55-497402c1809c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:09.354683 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf71c2-0f94-4bc0-bf55-497402c1809c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:09.354964 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf71c2-0f94-4bc0-bf55-497402c1809c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:09.394226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf71c2-0f94-4bc0-bf55-497402c1809c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:09.419287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-abbf71c2-0f94-4bc0-bf55-497402c1809c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:50:09.419287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:09.419287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:09.419287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:09.419287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:09.419287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:09.419287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:09.419287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:09.419287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:09.419287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:09.419287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:09.419287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:09.419287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:09.419287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:09.419287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:09.419287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:09.420977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:09.420977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:09.420977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:09.420977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:09.420977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:09.420977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:09.420977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:09.420977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:09.420977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:09.420977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:09.420977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:09.420977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:09.420977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:09.420977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:09.420977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:09.420977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:09.422571 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:09.422571 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:09.422571 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:09.422571 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:09.422571 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:09.422571 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:09.422571 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:09.422571 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:09.422571 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:09.422571 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:09.422571 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:50:09.422571 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:09.427386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 131/261] 10.222.0.76 () {56 vars in 1114 bytes} [Thu May 26 17:50:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 437 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:09.434945 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf71c2-0f94-4bc0-bf55-497402c1809c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:09.463349 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8a56c-0c7d-47f5-91e8-ac56d58f5a2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:09.464835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8a56c-0c7d-47f5-91e8-ac56d58f5a2f None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:09.519026 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:09.594714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:09.686465 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf71c2-0f94-4bc0-bf55-497402c1809c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:09.734834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 131/262] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:50:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 700 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:50:09.745122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8a56c-0c7d-47f5-91e8-ac56d58f5a2f None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:09.748397 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e493149-09b6-4bf9-ba10-57d0b14c3788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:09.749739 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e493149-09b6-4bf9-ba10-57d0b14c3788 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:09.750038 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e493149-09b6-4bf9-ba10-57d0b14c3788 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:09.750359 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e493149-09b6-4bf9-ba10-57d0b14c3788 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:09.751117 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e493149-09b6-4bf9-ba10-57d0b14c3788 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:09.751431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e493149-09b6-4bf9-ba10-57d0b14c3788 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:09.752813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e493149-09b6-4bf9-ba10-57d0b14c3788 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:09.771206 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 132/263] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:50:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:09.780181 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e493149-09b6-4bf9-ba10-57d0b14c3788 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:09.978301 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8a56c-0c7d-47f5-91e8-ac56d58f5a2f None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:09.979066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e493149-09b6-4bf9-ba10-57d0b14c3788 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:10.003496 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:10.022741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:10.044601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b03cd-8b0b-4668-bb14-0de459bb1ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:10.045742 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b03cd-8b0b-4668-bb14-0de459bb1ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:10.046082 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b03cd-8b0b-4668-bb14-0de459bb1ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:10.046424 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b03cd-8b0b-4668-bb14-0de459bb1ae6 None admin] PATH_INFO: `/v3/users/cb7c2cbcc6b843458bbdbf6998920f1e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:10.047547 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b03cd-8b0b-4668-bb14-0de459bb1ae6 None admin] RBAC: Authorizing `identity:get_user(user_id=cb7c2cbcc6b843458bbdbf6998920f1e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:10.055229 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b03cd-8b0b-4668-bb14-0de459bb1ae6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:10.057269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 133/264] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:50:09 2022] GET /identity/v3/users/cb7c2cbcc6b843458bbdbf6998920f1e => generated 283 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:10.064453 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f1877-3604-469b-bff1-80adda4fb824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:10.064877 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b03cd-8b0b-4668-bb14-0de459bb1ae6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:10.065325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f1877-3604-469b-bff1-80adda4fb824 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:10.065613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f1877-3604-469b-bff1-80adda4fb824 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:10.065892 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f1877-3604-469b-bff1-80adda4fb824 None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:10.071086 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f1877-3604-469b-bff1-80adda4fb824 None admin] RBAC: Authorizing `identity:get_project(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:10.072637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f1877-3604-469b-bff1-80adda4fb824 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:10.074413 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 132/265] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:09 2022] GET /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94 => generated 302 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:10.083330 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f1877-3604-469b-bff1-80adda4fb824 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:10.290546 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f1877-3604-469b-bff1-80adda4fb824 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:10.336798 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b03cd-8b0b-4668-bb14-0de459bb1ae6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:10.338440 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcab3c5-8229-4d54-ad6f-9f733fb8fd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:10.339111 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcab3c5-8229-4d54-ad6f-9f733fb8fd29 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:10.339397 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcab3c5-8229-4d54-ad6f-9f733fb8fd29 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:10.339679 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcab3c5-8229-4d54-ad6f-9f733fb8fd29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:10.340522 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcab3c5-8229-4d54-ad6f-9f733fb8fd29 None admin] RBAC: Adding query filter params (role.id=4fa8f06085c741a5b3cf64e63cc0754f, scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94, user.id=cb7c2cbcc6b843458bbdbf6998920f1e) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:10.340837 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcab3c5-8229-4d54-ad6f-9f733fb8fd29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:10.342333 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcab3c5-8229-4d54-ad6f-9f733fb8fd29 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:10.349940 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 133/266] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:50:10 2022] GET /identity/v3/role_assignments?user.id=cb7c2cbcc6b843458bbdbf6998920f1e&scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94&role.id=4fa8f06085c741a5b3cf64e63cc0754f => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:10.405992 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6500bf2-723c-45e6-8f3b-736091282439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:10.406978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6500bf2-723c-45e6-8f3b-736091282439 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:10.407446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6500bf2-723c-45e6-8f3b-736091282439 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:10.407872 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6500bf2-723c-45e6-8f3b-736091282439 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:10.475601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:10.504637 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6500bf2-723c-45e6-8f3b-736091282439 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:10.506142 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6500bf2-723c-45e6-8f3b-736091282439 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:10.649855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 134/267] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:50:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 17:50:10.654464 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcab3c5-8229-4d54-ad6f-9f733fb8fd29 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:10.722271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:10.781679 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcab3c5-8229-4d54-ad6f-9f733fb8fd29 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:10.817982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad6452-4d41-408b-b307-73008743dac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:10.818893 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad6452-4d41-408b-b307-73008743dac3 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:10.819180 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad6452-4d41-408b-b307-73008743dac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:10.819458 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad6452-4d41-408b-b307-73008743dac3 None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:10.820654 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad6452-4d41-408b-b307-73008743dac3 None admin] RBAC: Authorizing `identity:get_project(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:10.822099 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad6452-4d41-408b-b307-73008743dac3 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:10.824166 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 134/268] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:10 2022] GET /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:10.830013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6500bf2-723c-45e6-8f3b-736091282439 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:10.879716 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f041db-f194-4826-a019-725f4c50da55 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:10.880135 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f041db-f194-4826-a019-725f4c50da55 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:10.880415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f041db-f194-4826-a019-725f4c50da55 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:10.882120 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 135/269] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:10.888628 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f820dda-636f-4e93-acaf-3171ba213b65 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:10.888922 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f820dda-636f-4e93-acaf-3171ba213b65 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:10.889202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f820dda-636f-4e93-acaf-3171ba213b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:10.923272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-9f820dda-636f-4e93-acaf-3171ba213b65 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:10.924401 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-9f820dda-636f-4e93-acaf-3171ba213b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:10.932012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:10.959144 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6500bf2-723c-45e6-8f3b-736091282439 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:11.021360 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 136/270] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:50:11.048355 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ab0f5-906c-4cec-8973-d94b216f79de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:11.049105 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ab0f5-906c-4cec-8973-d94b216f79de None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:11.049395 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ab0f5-906c-4cec-8973-d94b216f79de None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:11.049720 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ab0f5-906c-4cec-8973-d94b216f79de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:11.050649 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ab0f5-906c-4cec-8973-d94b216f79de None admin] RBAC: Adding query filter params (role.id=e731efeb1d3c407fb58cf64a02d95796, scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94, user.id=cb7c2cbcc6b843458bbdbf6998920f1e) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:11.051502 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ab0f5-906c-4cec-8973-d94b216f79de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:11.053053 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ab0f5-906c-4cec-8973-d94b216f79de None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:11.060766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 135/271] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:50:10 2022] GET /identity/v3/role_assignments?user.id=cb7c2cbcc6b843458bbdbf6998920f1e&scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94&role.id=e731efeb1d3c407fb58cf64a02d95796 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:11.090604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f820dda-636f-4e93-acaf-3171ba213b65 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:11.151523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:11.233322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f820dda-636f-4e93-acaf-3171ba213b65 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:11.289422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfb621-9847-40c2-939a-7a3f5813fe35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:11.290879 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfb621-9847-40c2-939a-7a3f5813fe35 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:11.291402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfb621-9847-40c2-939a-7a3f5813fe35 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:11.291926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfb621-9847-40c2-939a-7a3f5813fe35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:11.299172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bfb621-9847-40c2-939a-7a3f5813fe35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:11.301746 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bfb621-9847-40c2-939a-7a3f5813fe35 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:11.303887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 137/272] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:11.309846 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ab0f5-906c-4cec-8973-d94b216f79de None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:11.311796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b63935-ebb9-4f55-a5b1-33463fb4126d None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:11.312090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b63935-ebb9-4f55-a5b1-33463fb4126d None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:11.312371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b63935-ebb9-4f55-a5b1-33463fb4126d None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:11.314066 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 138/273] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:11.321035 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a00c79-27b3-47dd-8136-2019c49c560d None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:11.321322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a00c79-27b3-47dd-8136-2019c49c560d None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:11.321600 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a00c79-27b3-47dd-8136-2019c49c560d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:11.341874 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-87a00c79-27b3-47dd-8136-2019c49c560d None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:11.343175 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-87a00c79-27b3-47dd-8136-2019c49c560d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:11.451445 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 139/274] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:50:11.475600 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a00c79-27b3-47dd-8136-2019c49c560d None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:11.483436 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ab0f5-906c-4cec-8973-d94b216f79de None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:11.528681 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4d52f-24cf-48c4-a9f1-05ea312f0156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:11.529733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4d52f-24cf-48c4-a9f1-05ea312f0156 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:11.530259 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4d52f-24cf-48c4-a9f1-05ea312f0156 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:11.530800 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4d52f-24cf-48c4-a9f1-05ea312f0156 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:11.531754 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4d52f-24cf-48c4-a9f1-05ea312f0156 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:11.533539 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4d52f-24cf-48c4-a9f1-05ea312f0156 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:11.539723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:11.571005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 136/275] 10.222.0.76 () {60 vars in 1132 bytes} [Thu May 26 17:50:11 2022] POST /identity/v3/users => generated 293 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:50:11.598642 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0711782-1366-48dc-bc26-afea839821ee None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:11.599028 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0711782-1366-48dc-bc26-afea839821ee None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:11.599521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0711782-1366-48dc-bc26-afea839821ee None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:11.607824 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 137/276] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:11.609973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f297d7-0c14-4cc1-a644-424bbc6e2049 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:11.610586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f297d7-0c14-4cc1-a644-424bbc6e2049 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:11.611030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f297d7-0c14-4cc1-a644-424bbc6e2049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:11.616381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a00c79-27b3-47dd-8136-2019c49c560d None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:11.617978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:11.635324 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-51f297d7-0c14-4cc1-a644-424bbc6e2049 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:11.638096 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-51f297d7-0c14-4cc1-a644-424bbc6e2049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:11.682629 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673fc19-5ccc-4845-8bef-4dd73fc7e816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:11.683760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673fc19-5ccc-4845-8bef-4dd73fc7e816 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:11.684234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673fc19-5ccc-4845-8bef-4dd73fc7e816 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:11.684673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673fc19-5ccc-4845-8bef-4dd73fc7e816 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:11.708283 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673fc19-5ccc-4845-8bef-4dd73fc7e816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:11.716723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-3673fc19-5ccc-4845-8bef-4dd73fc7e816 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:11.716723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:11.716723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:11.716723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:11.716723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:11.716723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:11.716723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:11.716723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:11.716723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:11.716723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:11.716723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:11.716723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:11.716723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:11.716723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:11.716723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:11.716723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:11.717992 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:11.717992 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:11.717992 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:11.717992 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:11.717992 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:11.717992 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:11.717992 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:11.717992 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:11.717992 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:11.717992 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:11.717992 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:11.717992 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:11.717992 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:11.717992 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:11.717992 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:11.717992 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:11.719042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:11.719042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:11.719042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:11.719042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:11.719042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:11.719042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:11.719042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:11.719042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:11.719042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:11.719042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:11.719042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:11.719042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:11.721535 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 140/277] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:11.730280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673fc19-5ccc-4845-8bef-4dd73fc7e816 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:11.744332 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 138/278] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:50:11.767402 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f297d7-0c14-4cc1-a644-424bbc6e2049 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:11.879448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673fc19-5ccc-4845-8bef-4dd73fc7e816 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:11.889456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:11.916859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f297d7-0c14-4cc1-a644-424bbc6e2049 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:11.931128 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347b7be-fd8d-4dd4-89c6-612f971c9117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:11.931816 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347b7be-fd8d-4dd4-89c6-612f971c9117 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:11.932099 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347b7be-fd8d-4dd4-89c6-612f971c9117 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:11.932394 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347b7be-fd8d-4dd4-89c6-612f971c9117 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:11.933187 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347b7be-fd8d-4dd4-89c6-612f971c9117 None admin] RBAC: Adding query filter params (name=member) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:11.933842 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347b7be-fd8d-4dd4-89c6-612f971c9117 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:11.935101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347b7be-fd8d-4dd4-89c6-612f971c9117 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:11.946407 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 141/279] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:50:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:11.947766 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5237c6c-8753-49fe-927d-6f28d4bafb78 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:11.948061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5237c6c-8753-49fe-927d-6f28d4bafb78 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:11.948337 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5237c6c-8753-49fe-927d-6f28d4bafb78 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:11.950005 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 142/280] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:11.956441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5237c6c-8753-49fe-927d-6f28d4bafb78 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:11.958721 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:11.965008 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7324a-f4ac-4a31-bfd5-028643d1464b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:11.966122 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7324a-f4ac-4a31-bfd5-028643d1464b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:11.966631 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7324a-f4ac-4a31-bfd5-028643d1464b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:11.967135 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7324a-f4ac-4a31-bfd5-028643d1464b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:11.985282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7324a-f4ac-4a31-bfd5-028643d1464b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:11.994078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-76c7324a-f4ac-4a31-bfd5-028643d1464b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:11.994078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:11.994078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:11.994078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:11.994078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:11.994078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:11.994078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:11.994078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:11.994078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:11.994078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:11.994078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:11.994078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:11.994078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:11.994078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:11.994078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:11.994078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:11.995069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:11.995069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:11.995069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:11.995069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:11.995069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:11.995069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:11.995069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:11.995069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:11.995069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:11.995069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:11.995069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:11.995069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:11.995069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:11.995069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:11.995069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:11.995069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:11.995880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:11.995880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:11.995880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:11.995880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:11.995880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:11.995880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:11.995880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:11.995880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:11.995880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:11.995880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:11.995880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:11.995880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:11.995880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 139/281] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:11.997064 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39037baa-ca06-497f-9009-6a690979a253 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:11.997226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39037baa-ca06-497f-9009-6a690979a253 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:11.997511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39037baa-ca06-497f-9009-6a690979a253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:12.016006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-39037baa-ca06-497f-9009-6a690979a253 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:12.017216 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-39037baa-ca06-497f-9009-6a690979a253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:12.114500 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5237c6c-8753-49fe-927d-6f28d4bafb78 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:12.120343 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 140/282] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:12.121381 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39037baa-ca06-497f-9009-6a690979a253 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:12.202347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f8627-7a5c-4984-8c3c-da4a5a438c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:12.203693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f8627-7a5c-4984-8c3c-da4a5a438c71 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:12.203980 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f8627-7a5c-4984-8c3c-da4a5a438c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:12.204312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f8627-7a5c-4984-8c3c-da4a5a438c71 None admin] PATH_INFO: `/v3/users/8f5da75ce6a34eb481d7db1e822ee078` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:12.232541 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f8627-7a5c-4984-8c3c-da4a5a438c71 None admin] RBAC: Authorizing `identity:get_user(user_id=8f5da75ce6a34eb481d7db1e822ee078)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:12.235672 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f8627-7a5c-4984-8c3c-da4a5a438c71 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:12.239839 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 143/283] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:50:11 2022] GET /identity/v3/users/8f5da75ce6a34eb481d7db1e822ee078 => generated 293 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:12.241596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f8627-7a5c-4984-8c3c-da4a5a438c71 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:12.247399 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39037baa-ca06-497f-9009-6a690979a253 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:12.274622 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:12.288513 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b0e92-1550-414a-9245-ca650056b07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:12.289207 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b0e92-1550-414a-9245-ca650056b07c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:12.289496 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b0e92-1550-414a-9245-ca650056b07c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:12.289775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b0e92-1550-414a-9245-ca650056b07c None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:12.290573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b0e92-1550-414a-9245-ca650056b07c None admin] RBAC: Adding query filter params (name=member) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:12.290882 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b0e92-1550-414a-9245-ca650056b07c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:12.292386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b0e92-1550-414a-9245-ca650056b07c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:12.302315 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 141/284] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:50:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:12.302925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:12.303303 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b0e92-1550-414a-9245-ca650056b07c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:12.415273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b0e92-1550-414a-9245-ca650056b07c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:12.450487 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f8627-7a5c-4984-8c3c-da4a5a438c71 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:12.453382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2745b-14a2-41fb-9593-0e5bd3178a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:12.454055 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2745b-14a2-41fb-9593-0e5bd3178a7b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:12.454396 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2745b-14a2-41fb-9593-0e5bd3178a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:12.454680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2745b-14a2-41fb-9593-0e5bd3178a7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:12.455525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2745b-14a2-41fb-9593-0e5bd3178a7b None admin] RBAC: Adding query filter params (role.id=eb1858a5621543028836223abcb3c0a1, user.id=8f5da75ce6a34eb481d7db1e822ee078, scope.system=all) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:12.455840 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2745b-14a2-41fb-9593-0e5bd3178a7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:12.457914 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2745b-14a2-41fb-9593-0e5bd3178a7b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:12.463710 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 142/285] 10.222.0.76 () {56 vars in 1308 bytes} [Thu May 26 17:50:12 2022] GET /identity/v3/role_assignments?user.id=8f5da75ce6a34eb481d7db1e822ee078&scope.system=all&role.id=eb1858a5621543028836223abcb3c0a1 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:12.464573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2745b-14a2-41fb-9593-0e5bd3178a7b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:12.498603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a465b-7ca7-4ac6-bd0e-04568e2fd5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:12.499384 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a465b-7ca7-4ac6-bd0e-04568e2fd5ce None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:12.499672 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a465b-7ca7-4ac6-bd0e-04568e2fd5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:12.499953 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a465b-7ca7-4ac6-bd0e-04568e2fd5ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:12.515721 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a465b-7ca7-4ac6-bd0e-04568e2fd5ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:12.524553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-860a465b-7ca7-4ac6-bd0e-04568e2fd5ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:50:12.524553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:12.524553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:12.524553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:12.524553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:12.524553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:12.524553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:12.524553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:12.524553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:12.524553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:12.524553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:12.524553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:12.524553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:12.524553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:12.524553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:12.524553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:12.525532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:12.525532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:12.525532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:12.525532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:12.525532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:12.525532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:12.525532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:12.525532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:12.525532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:12.525532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:12.525532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:12.525532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:12.525532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:12.525532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:12.525532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:12.525532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:12.526343 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:12.526343 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:12.526343 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:12.526343 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:12.526343 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:12.526343 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:12.526343 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:12.526343 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:12.526343 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:12.526343 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:12.526343 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:50:12.526343 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:12.526343 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 144/286] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:12.536790 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a465b-7ca7-4ac6-bd0e-04568e2fd5ce None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:12.562509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:12.728612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2745b-14a2-41fb-9593-0e5bd3178a7b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:12.763673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:12.790982 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e1d39-036e-406f-95e0-11f271195b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:12.792780 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e1d39-036e-406f-95e0-11f271195b95 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:12.793307 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e1d39-036e-406f-95e0-11f271195b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:12.794075 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e1d39-036e-406f-95e0-11f271195b95 None admin] PATH_INFO: `/v3/users/4d6bf94fec774561b130aa7ebf72152b` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:12.795623 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e1d39-036e-406f-95e0-11f271195b95 None admin] RBAC: Authorizing `identity:get_user(user_id=4d6bf94fec774561b130aa7ebf72152b)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:12.797140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a465b-7ca7-4ac6-bd0e-04568e2fd5ce None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:12.797528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e1d39-036e-406f-95e0-11f271195b95 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:12.799796 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 143/287] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:50:12 2022] GET /identity/v3/users/4d6bf94fec774561b130aa7ebf72152b => generated 297 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:12.808625 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e1d39-036e-406f-95e0-11f271195b95 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:12.856656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a4298-02a3-4989-99fc-acecd65f467b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:12.857790 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a4298-02a3-4989-99fc-acecd65f467b None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:12.858333 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a4298-02a3-4989-99fc-acecd65f467b None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:12.858844 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a4298-02a3-4989-99fc-acecd65f467b None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:12.860418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6a4298-02a3-4989-99fc-acecd65f467b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:12.860969 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6a4298-02a3-4989-99fc-acecd65f467b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:12.863705 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6a4298-02a3-4989-99fc-acecd65f467b None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:12.890122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 145/288] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:50:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:12.903755 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a4298-02a3-4989-99fc-acecd65f467b None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:13.128778 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e1d39-036e-406f-95e0-11f271195b95 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:13.171101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a4298-02a3-4989-99fc-acecd65f467b None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:13.173292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:13.214980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539cef8f-8d3b-477c-bd69-3ee40e71763f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:13.215646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cef8f-8d3b-477c-bd69-3ee40e71763f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:13.215930 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cef8f-8d3b-477c-bd69-3ee40e71763f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:13.216203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cef8f-8d3b-477c-bd69-3ee40e71763f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:13.235352 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95859d98-fae6-425b-8024-5142dac097d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:13.237561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95859d98-fae6-425b-8024-5142dac097d7 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:13.239033 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95859d98-fae6-425b-8024-5142dac097d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:13.239839 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95859d98-fae6-425b-8024-5142dac097d7 None admin] PATH_INFO: `/v3/users/c6e310df95684295b07dc76b8cc0c4b4` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:13.247577 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95859d98-fae6-425b-8024-5142dac097d7 None admin] RBAC: Authorizing `identity:get_user(user_id=c6e310df95684295b07dc76b8cc0c4b4)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:13.250875 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95859d98-fae6-425b-8024-5142dac097d7 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:13.262482 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 146/289] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:50:12 2022] GET /identity/v3/users/c6e310df95684295b07dc76b8cc0c4b4 => generated 297 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:13.264932 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95859d98-fae6-425b-8024-5142dac097d7 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:13.306996 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539cef8f-8d3b-477c-bd69-3ee40e71763f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:13.308546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539cef8f-8d3b-477c-bd69-3ee40e71763f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:13.333016 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:13.423023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 144/290] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:50:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 17:50:13.430992 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539cef8f-8d3b-477c-bd69-3ee40e71763f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:13.464136 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:13.566268 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95859d98-fae6-425b-8024-5142dac097d7 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:13.607099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539cef8f-8d3b-477c-bd69-3ee40e71763f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:13.651307 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64854790-b1e1-455e-81a4-54503fa1c61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:13.652782 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64854790-b1e1-455e-81a4-54503fa1c61e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:13.653732 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64854790-b1e1-455e-81a4-54503fa1c61e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:13.654518 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c045a-04d0-4424-b7a1-df45437ab571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:13.655555 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c045a-04d0-4424-b7a1-df45437ab571 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:13.655989 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64854790-b1e1-455e-81a4-54503fa1c61e None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:13.657105 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c045a-04d0-4424-b7a1-df45437ab571 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:13.657935 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64854790-b1e1-455e-81a4-54503fa1c61e None admin] RBAC: Authorizing `identity:get_project(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:13.658395 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c045a-04d0-4424-b7a1-df45437ab571 None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:13.661215 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64854790-b1e1-455e-81a4-54503fa1c61e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:13.662457 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2c045a-04d0-4424-b7a1-df45437ab571 None admin] RBAC: Authorizing `identity:get_project(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:13.664074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2c045a-04d0-4424-b7a1-df45437ab571 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:13.666066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 145/291] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:13 2022] GET /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:13.668150 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 147/292] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:13 2022] GET /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94 => generated 302 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:13.683216 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64854790-b1e1-455e-81a4-54503fa1c61e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:13.684383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c045a-04d0-4424-b7a1-df45437ab571 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:13.873125 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:13.894177 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64854790-b1e1-455e-81a4-54503fa1c61e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:13.909212 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:13.947495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370f53d-b22f-4dad-9a6c-065ead67035d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:13.948925 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370f53d-b22f-4dad-9a6c-065ead67035d None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:13.949392 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370f53d-b22f-4dad-9a6c-065ead67035d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:13.949855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370f53d-b22f-4dad-9a6c-065ead67035d None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94/users/c6e310df95684295b07dc76b8cc0c4b4/roles/5b3eae4902f44cc9a56bf2ec2a8e13f5` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:13.956032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370f53d-b22f-4dad-9a6c-065ead67035d None admin] RBAC: Authorizing `identity:create_grant(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94, user_id=c6e310df95684295b07dc76b8cc0c4b4, role_id=5b3eae4902f44cc9a56bf2ec2a8e13f5)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:13.959272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c045a-04d0-4424-b7a1-df45437ab571 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:13.959401 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370f53d-b22f-4dad-9a6c-065ead67035d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:14.001178 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 146/293] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:50:13 2022] PUT /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94/users/c6e310df95684295b07dc76b8cc0c4b4/roles/5b3eae4902f44cc9a56bf2ec2a8e13f5 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 17:50:14.056257 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54cb18-3537-400c-a763-c36363c553a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:14.057471 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54cb18-3537-400c-a763-c36363c553a4 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:14.057990 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54cb18-3537-400c-a763-c36363c553a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:14.058518 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54cb18-3537-400c-a763-c36363c553a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:14.060313 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54cb18-3537-400c-a763-c36363c553a4 None admin] RBAC: Adding query filter params (role.id=eb1858a5621543028836223abcb3c0a1, scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94, user.id=4d6bf94fec774561b130aa7ebf72152b) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:14.060898 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54cb18-3537-400c-a763-c36363c553a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:14.063928 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54cb18-3537-400c-a763-c36363c553a4 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:14.083842 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 148/294] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:50:13 2022] GET /identity/v3/role_assignments?user.id=4d6bf94fec774561b130aa7ebf72152b&scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94&role.id=eb1858a5621543028836223abcb3c0a1 => generated 608 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:14.279157 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc5ea6-da87-4747-939c-91466463a24c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:14.279735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc5ea6-da87-4747-939c-91466463a24c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:14.280137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc5ea6-da87-4747-939c-91466463a24c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:14.283427 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 147/295] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:14.291403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67291a-ee21-4f59-a9c7-6735c21d835d None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:14.291937 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67291a-ee21-4f59-a9c7-6735c21d835d None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:14.292399 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67291a-ee21-4f59-a9c7-6735c21d835d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:14.328725 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-0e67291a-ee21-4f59-a9c7-6735c21d835d None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:14.330946 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-0e67291a-ee21-4f59-a9c7-6735c21d835d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:14.477417 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 149/296] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:14.498734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e93d8d-0cff-4c1e-9e11-e3aa2fc94815 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:14.499231 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e93d8d-0cff-4c1e-9e11-e3aa2fc94815 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:14.499721 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e93d8d-0cff-4c1e-9e11-e3aa2fc94815 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:14.502299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 148/297] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:14.511574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f47df7-50fc-4ee4-9a68-c0933ea1956a None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:14.512079 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f47df7-50fc-4ee4-9a68-c0933ea1956a None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:14.512412 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f47df7-50fc-4ee4-9a68-c0933ea1956a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:14.515518 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e93d8d-0cff-4c1e-9e11-e3aa2fc94815 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:14.549124 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-77f47df7-50fc-4ee4-9a68-c0933ea1956a None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:14.550401 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-77f47df7-50fc-4ee4-9a68-c0933ea1956a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:14.667394 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 150/298] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:14.691231 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f47df7-50fc-4ee4-9a68-c0933ea1956a None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:14.702634 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:14.724851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e93d8d-0cff-4c1e-9e11-e3aa2fc94815 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:14.773914 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e6a02-aefd-4e5e-aef2-29b1bae101db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:14.775485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e6a02-aefd-4e5e-aef2-29b1bae101db None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:14.775777 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e6a02-aefd-4e5e-aef2-29b1bae101db None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:14.776063 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e6a02-aefd-4e5e-aef2-29b1bae101db None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:14.795605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e6a02-aefd-4e5e-aef2-29b1bae101db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:14.803814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-b94e6a02-aefd-4e5e-aef2-29b1bae101db None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:50:14.803814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:14.803814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:14.803814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:14.803814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:14.803814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:14.803814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:14.803814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:14.803814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:14.803814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:14.803814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:14.803814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:14.803814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:14.803814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:14.803814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:14.803814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:14.804785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:14.804785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:14.804785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:14.804785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:14.804785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:14.804785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:14.804785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:14.804785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:14.804785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:14.804785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:14.804785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:14.804785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:14.804785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:14.804785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:14.804785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:14.804785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:14.805584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:14.805584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:14.805584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:14.805584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:14.805584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:14.805584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:14.805584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:14.805584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:14.805584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:14.805584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:14.805584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:50:14.805584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:14.805584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 149/299] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:50:14.814235 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e6a02-aefd-4e5e-aef2-29b1bae101db None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:14.822744 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:14.848037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f47df7-50fc-4ee4-9a68-c0933ea1956a None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:14.922176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072a423-a1b5-42c4-b942-d616bb01353d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:14.923714 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072a423-a1b5-42c4-b942-d616bb01353d None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:14.924234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072a423-a1b5-42c4-b942-d616bb01353d None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:14.924746 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072a423-a1b5-42c4-b942-d616bb01353d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:14.927998 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072a423-a1b5-42c4-b942-d616bb01353d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:14.931226 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072a423-a1b5-42c4-b942-d616bb01353d None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:14.935358 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 151/300] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:14.936195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e6a02-aefd-4e5e-aef2-29b1bae101db None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:14.940964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072a423-a1b5-42c4-b942-d616bb01353d None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:14.977056 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f7ce8-912d-4e41-b864-bbe2e8765945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:14.978241 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f7ce8-912d-4e41-b864-bbe2e8765945 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:14.978710 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f7ce8-912d-4e41-b864-bbe2e8765945 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:14.979330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f7ce8-912d-4e41-b864-bbe2e8765945 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:14.980169 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f7ce8-912d-4e41-b864-bbe2e8765945 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:14.980552 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f7ce8-912d-4e41-b864-bbe2e8765945 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:14.982791 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f7ce8-912d-4e41-b864-bbe2e8765945 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:14.994659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 150/301] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:50:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:15.000271 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f7ce8-912d-4e41-b864-bbe2e8765945 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:15.041582 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:15.070898 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072a423-a1b5-42c4-b942-d616bb01353d None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:15.072102 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:15.111136 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f7ce8-912d-4e41-b864-bbe2e8765945 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:15.114332 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943020ee-1838-447b-9bfd-da011eb9eda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:15.115312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943020ee-1838-447b-9bfd-da011eb9eda1 None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:15.115796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943020ee-1838-447b-9bfd-da011eb9eda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:15.116267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943020ee-1838-447b-9bfd-da011eb9eda1 None admin] PATH_INFO: `/v3/groups` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:15.117180 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943020ee-1838-447b-9bfd-da011eb9eda1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:15.118891 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943020ee-1838-447b-9bfd-da011eb9eda1 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:15.168795 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 152/302] 10.222.0.76 () {60 vars in 1134 bytes} [Thu May 26 17:50:14 2022] POST /identity/v3/groups => generated 229 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:50:15.197097 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab56cf2-4e55-43ea-8b89-607da0fffa40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:15.199019 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab56cf2-4e55-43ea-8b89-607da0fffa40 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:15.199568 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab56cf2-4e55-43ea-8b89-607da0fffa40 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:15.200096 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab56cf2-4e55-43ea-8b89-607da0fffa40 None admin] PATH_INFO: `/v3/users/dc5a4de18f074fcea3c0e4eb4fd0eaef` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:15.230413 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab56cf2-4e55-43ea-8b89-607da0fffa40 None admin] RBAC: Authorizing `identity:get_user(user_id=dc5a4de18f074fcea3c0e4eb4fd0eaef)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:15.233457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab56cf2-4e55-43ea-8b89-607da0fffa40 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:15.237016 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 151/303] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:50:14 2022] GET /identity/v3/users/dc5a4de18f074fcea3c0e4eb4fd0eaef => generated 293 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:15.247395 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943020ee-1838-447b-9bfd-da011eb9eda1 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:15.415243 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943020ee-1838-447b-9bfd-da011eb9eda1 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:15.455227 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a42de9a-8bc4-41e7-b4da-563deb429bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:15.455900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42de9a-8bc4-41e7-b4da-563deb429bae None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:15.456188 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42de9a-8bc4-41e7-b4da-563deb429bae None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:15.456462 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42de9a-8bc4-41e7-b4da-563deb429bae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:15.457368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a42de9a-8bc4-41e7-b4da-563deb429bae None admin] RBAC: Adding query filter params (role.id=5b3eae4902f44cc9a56bf2ec2a8e13f5, user.id=dc5a4de18f074fcea3c0e4eb4fd0eaef, scope.system=all) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:15.458522 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a42de9a-8bc4-41e7-b4da-563deb429bae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:15.460005 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a42de9a-8bc4-41e7-b4da-563deb429bae None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:15.463842 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:15.471943 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 153/304] 10.222.0.76 () {56 vars in 1308 bytes} [Thu May 26 17:50:15 2022] GET /identity/v3/role_assignments?user.id=dc5a4de18f074fcea3c0e4eb4fd0eaef&scope.system=all&role.id=5b3eae4902f44cc9a56bf2ec2a8e13f5 => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:15.486661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a52d66-4cdc-4a77-b762-4cb24d1e48f6 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:15.487474 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a52d66-4cdc-4a77-b762-4cb24d1e48f6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:15.487617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a52d66-4cdc-4a77-b762-4cb24d1e48f6 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:15.491289 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 152/305] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:15.499372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f5946-ad02-46eb-997d-713d48d97f1e None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:15.499661 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f5946-ad02-46eb-997d-713d48d97f1e None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:15.499995 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f5946-ad02-46eb-997d-713d48d97f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:15.519902 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-e27f5946-ad02-46eb-997d-713d48d97f1e None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:15.521145 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-e27f5946-ad02-46eb-997d-713d48d97f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:15.607402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 154/306] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:50:15.651476 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a52d66-4cdc-4a77-b762-4cb24d1e48f6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:15.849681 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a52d66-4cdc-4a77-b762-4cb24d1e48f6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:15.901175 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:15.922903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae36110-c26b-4699-9bf9-8c84901c9834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:15.924533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae36110-c26b-4699-9bf9-8c84901c9834 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:15.925018 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae36110-c26b-4699-9bf9-8c84901c9834 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:15.926136 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae36110-c26b-4699-9bf9-8c84901c9834 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:15.949085 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae36110-c26b-4699-9bf9-8c84901c9834 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:15.965222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-aae36110-c26b-4699-9bf9-8c84901c9834 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:15.965222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:15.965222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:15.965222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:15.965222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:15.965222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:15.965222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:15.965222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:15.965222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:15.965222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:15.965222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:15.965222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:15.965222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:15.965222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:15.965222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:15.965222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:15.966903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:15.966903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:15.966903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:15.966903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:15.966903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:15.966903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:15.966903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:15.966903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:15.966903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:15.966903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:15.966903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:15.966903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:15.966903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:15.966903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:15.966903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:15.966903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:15.969227 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:15.969227 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:15.969227 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:15.969227 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:15.969227 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:15.969227 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:15.969227 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:15.969227 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:15.969227 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:15.969227 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:15.969227 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:15.969227 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:15.969227 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 153/307] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:15.978335 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f5946-ad02-46eb-997d-713d48d97f1e None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:16.167609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f5946-ad02-46eb-997d-713d48d97f1e None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:16.227016 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:16.234007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7710219-15fd-4ed2-a6b1-ed2cf6b01aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:16.235405 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7710219-15fd-4ed2-a6b1-ed2cf6b01aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:16.236157 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7710219-15fd-4ed2-a6b1-ed2cf6b01aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:16.237207 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7710219-15fd-4ed2-a6b1-ed2cf6b01aa0 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:16.238458 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7710219-15fd-4ed2-a6b1-ed2cf6b01aa0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:16.239005 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7710219-15fd-4ed2-a6b1-ed2cf6b01aa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:16.241597 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7710219-15fd-4ed2-a6b1-ed2cf6b01aa0 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:16.258945 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 155/308] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:50:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:16.265832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae36110-c26b-4699-9bf9-8c84901c9834 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:16.462350 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae36110-c26b-4699-9bf9-8c84901c9834 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:16.540679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8c330-38ce-4eee-a5cc-0d08f308c78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:16.542233 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8c330-38ce-4eee-a5cc-0d08f308c78d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:16.542751 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8c330-38ce-4eee-a5cc-0d08f308c78d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:16.543256 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8c330-38ce-4eee-a5cc-0d08f308c78d None admin] PATH_INFO: `/v3/users/8f5da75ce6a34eb481d7db1e822ee078` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:16.545131 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8c330-38ce-4eee-a5cc-0d08f308c78d None admin] RBAC: Authorizing `identity:get_user(user_id=8f5da75ce6a34eb481d7db1e822ee078)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:16.546658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8c330-38ce-4eee-a5cc-0d08f308c78d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:16.548354 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 154/309] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:50:16 2022] GET /identity/v3/users/8f5da75ce6a34eb481d7db1e822ee078 => generated 293 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:16.556257 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7710219-15fd-4ed2-a6b1-ed2cf6b01aa0 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:16.710654 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7710219-15fd-4ed2-a6b1-ed2cf6b01aa0 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:16.757271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c09c548-3c45-487d-a4d4-03b6e78e8101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:16.758458 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09c548-3c45-487d-a4d4-03b6e78e8101 None admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:16.758833 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09c548-3c45-487d-a4d4-03b6e78e8101 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:16.759154 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09c548-3c45-487d-a4d4-03b6e78e8101 None admin] PATH_INFO: `/v3/system/users/8f5da75ce6a34eb481d7db1e822ee078/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:16.760893 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c09c548-3c45-487d-a4d4-03b6e78e8101 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8f5da75ce6a34eb481d7db1e822ee078, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:16.762640 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c09c548-3c45-487d-a4d4-03b6e78e8101 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:16.774003 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 156/310] 10.222.0.76 () {58 vars in 1334 bytes} [Thu May 26 17:50:16 2022] PUT /identity/v3/system/users/8f5da75ce6a34eb481d7db1e822ee078/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 17:50:17.206150 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41927dec-aa6a-4c46-b0f3-b23922d6adb0 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:17.206511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41927dec-aa6a-4c46-b0f3-b23922d6adb0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:17.206793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41927dec-aa6a-4c46-b0f3-b23922d6adb0 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:17.208794 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 155/311] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:17.214962 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a263fd-1279-47fa-96d4-f7742c19b536 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:17.215257 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a263fd-1279-47fa-96d4-f7742c19b536 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:17.215542 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a263fd-1279-47fa-96d4-f7742c19b536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:17.234826 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-b4a263fd-1279-47fa-96d4-f7742c19b536 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:17.236106 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-b4a263fd-1279-47fa-96d4-f7742c19b536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:17.307097 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:17.330582 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 157/312] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:17.355508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41927dec-aa6a-4c46-b0f3-b23922d6adb0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:17.423652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:17.485702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41927dec-aa6a-4c46-b0f3-b23922d6adb0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:17.526983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9e6be-0159-48f1-98d5-aaf918841d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:17.527797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9e6be-0159-48f1-98d5-aaf918841d74 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:17.528083 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9e6be-0159-48f1-98d5-aaf918841d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:17.528357 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9e6be-0159-48f1-98d5-aaf918841d74 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:17.541868 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9e6be-0159-48f1-98d5-aaf918841d74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:17.549790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-4cd9e6be-0159-48f1-98d5-aaf918841d74 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:50:17.549790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:17.549790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:17.549790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:17.549790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:17.549790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:17.549790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:17.549790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:17.549790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:17.549790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:17.549790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:17.549790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:17.549790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:17.549790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:17.549790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:17.549790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:17.551788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:17.551788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:17.551788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:17.551788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:17.551788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:17.551788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:17.551788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:17.551788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:17.551788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:17.551788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:17.551788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:17.551788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:17.551788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:17.551788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:17.551788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:17.551788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:17.552950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:17.552950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:17.552950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:17.552950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:17.552950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:17.552950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:17.552950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:17.552950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:17.552950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:17.552950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:17.552950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:50:17.552950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:17.552950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 156/313] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:17.557370 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a263fd-1279-47fa-96d4-f7742c19b536 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:17.686925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a263fd-1279-47fa-96d4-f7742c19b536 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:17.727716 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7bb07-8f60-48da-abc4-48aa567b01d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:17.729043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7bb07-8f60-48da-abc4-48aa567b01d5 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:17.729546 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7bb07-8f60-48da-abc4-48aa567b01d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:17.730127 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7bb07-8f60-48da-abc4-48aa567b01d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:17.731148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7bb07-8f60-48da-abc4-48aa567b01d5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:17.731667 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7bb07-8f60-48da-abc4-48aa567b01d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:17.733371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7bb07-8f60-48da-abc4-48aa567b01d5 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:17.744183 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 158/314] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:50:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:17.751569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9e6be-0159-48f1-98d5-aaf918841d74 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:17.876009 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9e6be-0159-48f1-98d5-aaf918841d74 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:17.895482 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:17.923891 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e0605-87df-4b2f-ba70-090d6d33ba91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:17.924791 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e0605-87df-4b2f-ba70-090d6d33ba91 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:17.925078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e0605-87df-4b2f-ba70-090d6d33ba91 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:17.925356 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e0605-87df-4b2f-ba70-090d6d33ba91 None admin] PATH_INFO: `/v3/users/c6e310df95684295b07dc76b8cc0c4b4` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:17.927633 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e0605-87df-4b2f-ba70-090d6d33ba91 None admin] RBAC: Authorizing `identity:get_user(user_id=c6e310df95684295b07dc76b8cc0c4b4)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:17.929117 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e0605-87df-4b2f-ba70-090d6d33ba91 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:17.930882 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 157/315] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:50:17 2022] GET /identity/v3/users/c6e310df95684295b07dc76b8cc0c4b4 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:17.937622 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7bb07-8f60-48da-abc4-48aa567b01d5 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:17.999371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:18.059197 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7bb07-8f60-48da-abc4-48aa567b01d5 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:18.083659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dccaf0-b07f-4f6a-8024-f5be105cadd3 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:18.083993 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dccaf0-b07f-4f6a-8024-f5be105cadd3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:18.084232 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dccaf0-b07f-4f6a-8024-f5be105cadd3 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:18.089811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 158/316] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:18.091675 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7d681-cc13-4580-bf36-53e1bcb7be32 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:18.091964 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7d681-cc13-4580-bf36-53e1bcb7be32 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:18.092240 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7d681-cc13-4580-bf36-53e1bcb7be32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:18.100383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482008b-f128-4805-91db-8ee224cae948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:18.101064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482008b-f128-4805-91db-8ee224cae948 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:18.101353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482008b-f128-4805-91db-8ee224cae948 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:18.101639 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482008b-f128-4805-91db-8ee224cae948 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:18.111155 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e7f7d681-cc13-4580-bf36-53e1bcb7be32 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:18.113507 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e7f7d681-cc13-4580-bf36-53e1bcb7be32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:18.190062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c482008b-f128-4805-91db-8ee224cae948 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:18.192094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c482008b-f128-4805-91db-8ee224cae948 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:18.226361 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 159/317] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:18.250504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7d681-cc13-4580-bf36-53e1bcb7be32 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:18.257882 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:18.342138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 159/318] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:50:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:50:18.349214 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482008b-f128-4805-91db-8ee224cae948 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:18.415159 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7d681-cc13-4580-bf36-53e1bcb7be32 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:18.432720 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:18.463095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a72f9-51c7-44b1-af04-9d06c2af2713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:18.464199 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a72f9-51c7-44b1-af04-9d06c2af2713 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:18.464665 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a72f9-51c7-44b1-af04-9d06c2af2713 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:18.465122 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a72f9-51c7-44b1-af04-9d06c2af2713 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:18.466587 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a72f9-51c7-44b1-af04-9d06c2af2713 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:18.468969 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a72f9-51c7-44b1-af04-9d06c2af2713 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:18.470927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 160/319] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:18.473633 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ee5be-7d8e-4c9a-8632-57702bfd67b8 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:18.474141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ee5be-7d8e-4c9a-8632-57702bfd67b8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:18.474629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ee5be-7d8e-4c9a-8632-57702bfd67b8 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:18.476816 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 161/320] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:18.478971 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ee5be-7d8e-4c9a-8632-57702bfd67b8 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:18.514182 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482008b-f128-4805-91db-8ee224cae948 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:18.578900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6607c6-e0b2-4292-bb21-7973e86f9e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:18.581300 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6607c6-e0b2-4292-bb21-7973e86f9e73 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:18.582305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6607c6-e0b2-4292-bb21-7973e86f9e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:18.582930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6607c6-e0b2-4292-bb21-7973e86f9e73 None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:18.585320 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6607c6-e0b2-4292-bb21-7973e86f9e73 None admin] RBAC: Authorizing `identity:get_project(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:18.588133 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6607c6-e0b2-4292-bb21-7973e86f9e73 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:18.591288 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 160/321] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:18 2022] GET /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:18.593853 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6b4b1-c655-45c5-9bff-27f26360ac30 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:18.594401 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6b4b1-c655-45c5-9bff-27f26360ac30 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:18.594934 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6b4b1-c655-45c5-9bff-27f26360ac30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:18.611444 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ee5be-7d8e-4c9a-8632-57702bfd67b8 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:18.621318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-28d6b4b1-c655-45c5-9bff-27f26360ac30 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:18.622629 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-28d6b4b1-c655-45c5-9bff-27f26360ac30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:18.637524 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:18.667626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe24adf-ca91-4652-9c17-be6692186b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:18.668537 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe24adf-ca91-4652-9c17-be6692186b49 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:18.668963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe24adf-ca91-4652-9c17-be6692186b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:18.669386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe24adf-ca91-4652-9c17-be6692186b49 None admin] PATH_INFO: `/v3/groups` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:18.670656 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe24adf-ca91-4652-9c17-be6692186b49 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:18.672624 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe24adf-ca91-4652-9c17-be6692186b49 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:18.696429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 162/322] 10.222.0.76 () {60 vars in 1134 bytes} [Thu May 26 17:50:18 2022] POST /identity/v3/groups => generated 226 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:50:18.704479 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe24adf-ca91-4652-9c17-be6692186b49 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:18.794434 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 161/323] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:18.820591 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:18.827361 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6b4b1-c655-45c5-9bff-27f26360ac30 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:18.959276 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe24adf-ca91-4652-9c17-be6692186b49 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:19.035001 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305544a-220e-4774-b13c-bc348077dc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:19.037682 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305544a-220e-4774-b13c-bc348077dc09 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:19.038462 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305544a-220e-4774-b13c-bc348077dc09 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:19.039053 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305544a-220e-4774-b13c-bc348077dc09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:19.041231 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305544a-220e-4774-b13c-bc348077dc09 None admin] RBAC: Adding query filter params (role.id=5b3eae4902f44cc9a56bf2ec2a8e13f5, scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94, user.id=c6e310df95684295b07dc76b8cc0c4b4) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:19.042116 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305544a-220e-4774-b13c-bc348077dc09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:19.045230 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305544a-220e-4774-b13c-bc348077dc09 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:19.059615 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 163/324] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:50:18 2022] GET /identity/v3/role_assignments?user.id=c6e310df95684295b07dc76b8cc0c4b4&scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94&role.id=5b3eae4902f44cc9a56bf2ec2a8e13f5 => generated 608 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:19.076450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6b4b1-c655-45c5-9bff-27f26360ac30 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:19.126561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97387835-08e9-4a42-b6b3-fb96d9dc24ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:19.127444 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97387835-08e9-4a42-b6b3-fb96d9dc24ae None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:19.127740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97387835-08e9-4a42-b6b3-fb96d9dc24ae None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:19.128068 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97387835-08e9-4a42-b6b3-fb96d9dc24ae None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:19.147772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97387835-08e9-4a42-b6b3-fb96d9dc24ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:19.156240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-97387835-08e9-4a42-b6b3-fb96d9dc24ae None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:50:19.156240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:19.156240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:19.156240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:19.156240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:19.156240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:19.156240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:19.156240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:19.156240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:19.156240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:19.156240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:19.156240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:19.156240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:19.156240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:19.156240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:19.156240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:19.157421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:19.157421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:19.157421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:19.157421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:19.157421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:19.157421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:19.157421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:19.157421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:19.157421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:19.157421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:19.157421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:19.157421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:19.157421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:19.157421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:19.157421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:19.157421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:19.158437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:19.158437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:19.158437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:19.158437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:19.158437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:19.158437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:19.158437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:19.158437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:19.158437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:19.158437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:19.158437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:50:19.158437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:19.158437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 162/325] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:19.168454 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305544a-220e-4774-b13c-bc348077dc09 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:19.408785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305544a-220e-4774-b13c-bc348077dc09 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:19.412357 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:19.480064 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947e011-245e-4021-a049-e0913ca5ad37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:19.481234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947e011-245e-4021-a049-e0913ca5ad37 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:19.481717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947e011-245e-4021-a049-e0913ca5ad37 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:19.482225 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947e011-245e-4021-a049-e0913ca5ad37 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:19.483720 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947e011-245e-4021-a049-e0913ca5ad37 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:19.484284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947e011-245e-4021-a049-e0913ca5ad37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:19.487020 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947e011-245e-4021-a049-e0913ca5ad37 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:19.506744 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 164/326] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:50:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:19.515641 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97387835-08e9-4a42-b6b3-fb96d9dc24ae None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:19.530127 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:19.539772 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7fe64-3d52-4f6c-bab3-466af6ad9512 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:19.540860 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7fe64-3d52-4f6c-bab3-466af6ad9512 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:19.541701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7fe64-3d52-4f6c-bab3-466af6ad9512 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:19.545328 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 165/327] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:19.555804 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb06d6-fb89-4304-ae4f-532ed9179385 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:19.556828 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb06d6-fb89-4304-ae4f-532ed9179385 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:19.557586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb06d6-fb89-4304-ae4f-532ed9179385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:19.591309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-48bb06d6-fb89-4304-ae4f-532ed9179385 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:19.594018 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-48bb06d6-fb89-4304-ae4f-532ed9179385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:19.656080 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97387835-08e9-4a42-b6b3-fb96d9dc24ae None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:19.705606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb8281-c803-4a0b-a7a0-c71471ceceef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:19.706469 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb8281-c803-4a0b-a7a0-c71471ceceef None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:19.707016 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb8281-c803-4a0b-a7a0-c71471ceceef None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:19.708155 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb8281-c803-4a0b-a7a0-c71471ceceef None admin] PATH_INFO: `/v3/users/dc5a4de18f074fcea3c0e4eb4fd0eaef` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:19.709554 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb8281-c803-4a0b-a7a0-c71471ceceef None admin] RBAC: Authorizing `identity:get_user(user_id=dc5a4de18f074fcea3c0e4eb4fd0eaef)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:19.711976 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb8281-c803-4a0b-a7a0-c71471ceceef None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:19.714950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 163/328] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:50:19 2022] GET /identity/v3/users/dc5a4de18f074fcea3c0e4eb4fd0eaef => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:19.727510 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb8281-c803-4a0b-a7a0-c71471ceceef None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:19.771848 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 166/329] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:19.817272 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb06d6-fb89-4304-ae4f-532ed9179385 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:19.871713 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:19.926625 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb8281-c803-4a0b-a7a0-c71471ceceef None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:19.995146 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:20.002355 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb06d6-fb89-4304-ae4f-532ed9179385 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:20.006933 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52046dfe-9d59-4898-ab5b-701843952f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:20.008163 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52046dfe-9d59-4898-ab5b-701843952f0c None admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:20.008678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52046dfe-9d59-4898-ab5b-701843952f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:20.009184 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52046dfe-9d59-4898-ab5b-701843952f0c None admin] PATH_INFO: `/v3/system/users/dc5a4de18f074fcea3c0e4eb4fd0eaef/roles/5b3eae4902f44cc9a56bf2ec2a8e13f5` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:20.016242 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52046dfe-9d59-4898-ab5b-701843952f0c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=dc5a4de18f074fcea3c0e4eb4fd0eaef, role_id=5b3eae4902f44cc9a56bf2ec2a8e13f5)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:20.018032 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52046dfe-9d59-4898-ab5b-701843952f0c None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:20.031372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 164/330] 10.222.0.76 () {58 vars in 1334 bytes} [Thu May 26 17:50:19 2022] PUT /identity/v3/system/users/dc5a4de18f074fcea3c0e4eb4fd0eaef/roles/5b3eae4902f44cc9a56bf2ec2a8e13f5 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 17:50:20.059751 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf4cba-f1c2-47a9-98ee-f37dfda1b50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:20.060967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf4cba-f1c2-47a9-98ee-f37dfda1b50d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:20.061467 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf4cba-f1c2-47a9-98ee-f37dfda1b50d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:20.061951 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf4cba-f1c2-47a9-98ee-f37dfda1b50d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:20.080031 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf4cba-f1c2-47a9-98ee-f37dfda1b50d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:20.088669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-2bdf4cba-f1c2-47a9-98ee-f37dfda1b50d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:20.088669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:20.088669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:20.088669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:20.088669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:20.088669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:20.088669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:20.088669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:20.088669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:20.088669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:20.088669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:20.088669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:20.088669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:20.088669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:20.088669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:20.088669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:20.089645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:20.089645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:20.089645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:20.089645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:20.089645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:20.089645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:20.089645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:20.089645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:20.089645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:20.089645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:20.089645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:20.089645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:20.089645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:20.089645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:20.089645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:20.089645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:20.090517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:20.090517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:20.090517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:20.090517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:20.090517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:20.090517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:20.090517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:20.090517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:20.090517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:20.090517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:20.090517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:20.090517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:20.093540 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 167/331] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:20.101803 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52046dfe-9d59-4898-ab5b-701843952f0c None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:20.419842 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52046dfe-9d59-4898-ab5b-701843952f0c None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:20.494202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd742c8b-34c0-49c1-bb30-744bacfcc25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:20.495371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd742c8b-34c0-49c1-bb30-744bacfcc25f None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:20.495902 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd742c8b-34c0-49c1-bb30-744bacfcc25f None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:20.496400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd742c8b-34c0-49c1-bb30-744bacfcc25f None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:20.497882 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd742c8b-34c0-49c1-bb30-744bacfcc25f None admin] RBAC: Adding query filter params (name=member) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:20.498499 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd742c8b-34c0-49c1-bb30-744bacfcc25f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:20.501222 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd742c8b-34c0-49c1-bb30-744bacfcc25f None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:20.519940 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 165/332] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:50:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:20.520655 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:20.531208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf4cba-f1c2-47a9-98ee-f37dfda1b50d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:20.700087 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf4cba-f1c2-47a9-98ee-f37dfda1b50d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:20.767803 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f9bd4-862e-49e3-acde-740d959e898b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:20.769916 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f9bd4-862e-49e3-acde-740d959e898b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:20.770451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f9bd4-862e-49e3-acde-740d959e898b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:20.770952 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f9bd4-862e-49e3-acde-740d959e898b None admin] PATH_INFO: `/v3/users/8f5da75ce6a34eb481d7db1e822ee078` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:20.773203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f9bd4-862e-49e3-acde-740d959e898b None admin] RBAC: Authorizing `identity:get_user(user_id=8f5da75ce6a34eb481d7db1e822ee078)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:20.775677 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f9bd4-862e-49e3-acde-740d959e898b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:20.779421 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 168/333] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:50:20 2022] GET /identity/v3/users/8f5da75ce6a34eb481d7db1e822ee078 => generated 293 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:20.787034 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd742c8b-34c0-49c1-bb30-744bacfcc25f None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:21.038906 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd742c8b-34c0-49c1-bb30-744bacfcc25f None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:21.115225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b7006f-f4d4-4edd-adcc-011d89e04735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:21.116341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7006f-f4d4-4edd-adcc-011d89e04735 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:21.116816 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7006f-f4d4-4edd-adcc-011d89e04735 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:21.117363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7006f-f4d4-4edd-adcc-011d89e04735 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:21.118793 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b7006f-f4d4-4edd-adcc-011d89e04735 None admin] RBAC: Adding query filter params (role.id=eb1858a5621543028836223abcb3c0a1, user.id=8f5da75ce6a34eb481d7db1e822ee078, scope.system=all) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:21.119120 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b7006f-f4d4-4edd-adcc-011d89e04735 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:21.120809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b7006f-f4d4-4edd-adcc-011d89e04735 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:21.140054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 166/334] 10.222.0.76 () {56 vars in 1308 bytes} [Thu May 26 17:50:20 2022] GET /identity/v3/role_assignments?user.id=8f5da75ce6a34eb481d7db1e822ee078&scope.system=all&role.id=eb1858a5621543028836223abcb3c0a1 => generated 510 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:22.774030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e6698-48f4-4021-afa7-f4b8b8396cb3 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:22.774629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e6698-48f4-4021-afa7-f4b8b8396cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:22.774629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e6698-48f4-4021-afa7-f4b8b8396cb3 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:22.776440 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 169/335] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:22.787872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad29a61-de10-4d11-9355-c89f73fc29d6 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:22.789210 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad29a61-de10-4d11-9355-c89f73fc29d6 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:22.789964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad29a61-de10-4d11-9355-c89f73fc29d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:22.815274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-5ad29a61-de10-4d11-9355-c89f73fc29d6 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:22.816936 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-5ad29a61-de10-4d11-9355-c89f73fc29d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:22.881021 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:22.899855 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 167/336] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:22.936065 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad29a61-de10-4d11-9355-c89f73fc29d6 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:22.936487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ea8a0-3aeb-4720-9623-23e9fb409b4a None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:22.936780 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ea8a0-3aeb-4720-9623-23e9fb409b4a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:22.937062 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ea8a0-3aeb-4720-9623-23e9fb409b4a None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:22.938538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 170/337] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:22.945044 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db3d04-fdee-4d97-b039-625bef56d9e1 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:22.945358 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db3d04-fdee-4d97-b039-625bef56d9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:22.945606 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db3d04-fdee-4d97-b039-625bef56d9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:22.965527 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-55db3d04-fdee-4d97-b039-625bef56d9e1 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:22.966920 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-55db3d04-fdee-4d97-b039-625bef56d9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:22.987507 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:23.050588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 171/338] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:50:23.071474 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db3d04-fdee-4d97-b039-625bef56d9e1 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:23.172607 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad29a61-de10-4d11-9355-c89f73fc29d6 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:23.198561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db3d04-fdee-4d97-b039-625bef56d9e1 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:23.244588 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069b35e-b58d-47c3-b9c4-965d00eb5a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:23.245687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069b35e-b58d-47c3-b9c4-965d00eb5a81 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:23.246127 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069b35e-b58d-47c3-b9c4-965d00eb5a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:23.246572 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069b35e-b58d-47c3-b9c4-965d00eb5a81 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:23.258516 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8f3a2-aac6-484c-af1d-6642c584a2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:23.260391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8f3a2-aac6-484c-af1d-6642c584a2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:23.261197 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8f3a2-aac6-484c-af1d-6642c584a2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:23.261977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8f3a2-aac6-484c-af1d-6642c584a2c2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:23.275230 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069b35e-b58d-47c3-b9c4-965d00eb5a81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:23.282897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8f3a2-aac6-484c-af1d-6642c584a2c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:23.283733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-2069b35e-b58d-47c3-b9c4-965d00eb5a81 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:23.283733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:23.283733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:23.283733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:23.283733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:23.283733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:23.283733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:23.283733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:23.283733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:23.283733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:23.283733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:23.283733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:23.283733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:23.283733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:23.283733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:23.283733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:23.286087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:23.286087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:23.286087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:23.286087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:23.286087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:23.286087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:23.286087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:23.286087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:23.286087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:23.286087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:23.286087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:23.286087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:23.286087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:23.286087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:23.286087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:23.286087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:23.287957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:23.287957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:23.287957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:23.287957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:23.287957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:23.287957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:23.287957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:23.287957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:23.287957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:23.287957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:23.287957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:23.287957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:23.297958 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 168/339] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:23.314485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-b3d8f3a2-aac6-484c-af1d-6642c584a2c2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:50:23.314485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:23.314485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:23.314485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:23.314485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:23.314485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:23.314485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:23.314485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:23.314485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:23.314485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:23.314485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:23.314485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:23.314485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:23.314485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:23.314485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:23.314485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:23.316735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:23.316735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:23.316735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:23.316735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:23.316735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:23.316735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:23.316735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:23.316735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:23.316735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:23.316735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:23.316735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:23.316735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:23.316735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:23.316735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:23.316735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:23.316735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:23.318408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:23.318408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:23.318408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:23.318408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:23.318408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:23.318408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:23.318408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:23.318408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:23.318408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:23.318408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:23.318408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:50:23.318408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:23.322872 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 172/340] 10.222.0.76 () {56 vars in 1114 bytes} [Thu May 26 17:50:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:50:23.332013 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069b35e-b58d-47c3-b9c4-965d00eb5a81 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:23.337014 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8f3a2-aac6-484c-af1d-6642c584a2c2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:23.371322 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:23.456312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:23.555503 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8f3a2-aac6-484c-af1d-6642c584a2c2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:23.602427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069b35e-b58d-47c3-b9c4-965d00eb5a81 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:23.625582 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadd10c-b2f6-4eb4-a5e5-790f9257bb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:23.627455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadd10c-b2f6-4eb4-a5e5-790f9257bb78 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:23.628248 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadd10c-b2f6-4eb4-a5e5-790f9257bb78 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:23.629672 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadd10c-b2f6-4eb4-a5e5-790f9257bb78 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:23.631622 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadd10c-b2f6-4eb4-a5e5-790f9257bb78 None admin] RBAC: Adding query filter params (name=member) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:23.632523 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadd10c-b2f6-4eb4-a5e5-790f9257bb78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:23.635611 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadd10c-b2f6-4eb4-a5e5-790f9257bb78 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:23.656313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 173/341] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:50:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:23.663266 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738ab62-e617-49e9-b59a-f5c8ce337f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:23.664252 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738ab62-e617-49e9-b59a-f5c8ce337f4d None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:23.664674 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadd10c-b2f6-4eb4-a5e5-790f9257bb78 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:23.664956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738ab62-e617-49e9-b59a-f5c8ce337f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:23.665415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738ab62-e617-49e9-b59a-f5c8ce337f4d None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:23.666394 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1738ab62-e617-49e9-b59a-f5c8ce337f4d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:23.666874 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1738ab62-e617-49e9-b59a-f5c8ce337f4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:23.668496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1738ab62-e617-49e9-b59a-f5c8ce337f4d None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:23.683473 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 169/342] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:50:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:23.708374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738ab62-e617-49e9-b59a-f5c8ce337f4d None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:23.837850 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:23.872666 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738ab62-e617-49e9-b59a-f5c8ce337f4d None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:23.893781 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:23.893973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadd10c-b2f6-4eb4-a5e5-790f9257bb78 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:23.950940 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d433a-8973-42e5-9720-eea357e367ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:23.952254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d433a-8973-42e5-9720-eea357e367ae None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:23.952931 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d433a-8973-42e5-9720-eea357e367ae None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:23.953608 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d433a-8973-42e5-9720-eea357e367ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:23.968625 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c317c-97f0-4707-b6d8-5930d1ff7915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:23.969672 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c317c-97f0-4707-b6d8-5930d1ff7915 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:23.970151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c317c-97f0-4707-b6d8-5930d1ff7915 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:23.970636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c317c-97f0-4707-b6d8-5930d1ff7915 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:24.110171 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c317c-97f0-4707-b6d8-5930d1ff7915 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:24.113547 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c317c-97f0-4707-b6d8-5930d1ff7915 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:24.159853 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d433a-8973-42e5-9720-eea357e367ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:24.162838 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d433a-8973-42e5-9720-eea357e367ae None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:24.219422 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:24.273276 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 174/343] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:50:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 17:50:24.280817 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c317c-97f0-4707-b6d8-5930d1ff7915 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:24.384713 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 170/344] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:50:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 677 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:50:24.392862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d433a-8973-42e5-9720-eea357e367ae None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:24.422125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:24.572522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c317c-97f0-4707-b6d8-5930d1ff7915 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:24.627875 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8ce98-b683-43cb-9085-fe3eafd85cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:24.629059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8ce98-b683-43cb-9085-fe3eafd85cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:24.629526 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8ce98-b683-43cb-9085-fe3eafd85cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:24.629954 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8ce98-b683-43cb-9085-fe3eafd85cc0 None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:24.631659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b8ce98-b683-43cb-9085-fe3eafd85cc0 None admin] RBAC: Authorizing `identity:get_project(project_id=89f44d18dc9b464faeb1bf0bd03569af)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:24.633375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b8ce98-b683-43cb-9085-fe3eafd85cc0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:24.640428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 175/345] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:24 2022] GET /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af => generated 298 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:24.642048 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8ce98-b683-43cb-9085-fe3eafd85cc0 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:24.649603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d433a-8973-42e5-9720-eea357e367ae None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:24.717117 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:24.720414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9220f-a8e4-4a1c-b6d0-9108f8472fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:24.723935 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9220f-a8e4-4a1c-b6d0-9108f8472fba None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:24.724567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9220f-a8e4-4a1c-b6d0-9108f8472fba None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:24.725136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9220f-a8e4-4a1c-b6d0-9108f8472fba None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:24.731240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9220f-a8e4-4a1c-b6d0-9108f8472fba None admin] RBAC: Authorizing `identity:get_project(project_id=89f44d18dc9b464faeb1bf0bd03569af)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:24.733169 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9220f-a8e4-4a1c-b6d0-9108f8472fba None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:24.735679 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 171/346] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:24 2022] GET /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af => generated 298 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:24.741728 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9220f-a8e4-4a1c-b6d0-9108f8472fba None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:24.784258 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8ce98-b683-43cb-9085-fe3eafd85cc0 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:24.827083 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a427c-7781-44db-b06f-eee531a41a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:24.828240 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a427c-7781-44db-b06f-eee531a41a3d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:24.828724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a427c-7781-44db-b06f-eee531a41a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:24.829156 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a427c-7781-44db-b06f-eee531a41a3d None admin] PATH_INFO: `/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:24.837098 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a427c-7781-44db-b06f-eee531a41a3d None admin] RBAC: Authorizing `identity:get_group(group_id=93dde9ef534d4661a2359bb5d1ea03f5)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:24.838875 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a427c-7781-44db-b06f-eee531a41a3d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:24.846361 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 176/347] 10.222.0.76 () {56 vars in 1180 bytes} [Thu May 26 17:50:24 2022] GET /identity/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:24.849407 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc838b8-972f-4944-aee2-daaf26b87aaa None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:24.849916 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc838b8-972f-4944-aee2-daaf26b87aaa None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:24.854572 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc838b8-972f-4944-aee2-daaf26b87aaa None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:24.861467 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 177/348] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:24 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:24.861467 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc838b8-972f-4944-aee2-daaf26b87aaa None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:24.891457 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:24.945933 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9220f-a8e4-4a1c-b6d0-9108f8472fba None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:25.029479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643612d-3a0e-4cea-8a92-150c76a72385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:25.030707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643612d-3a0e-4cea-8a92-150c76a72385 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:25.031000 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643612d-3a0e-4cea-8a92-150c76a72385 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:25.031301 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643612d-3a0e-4cea-8a92-150c76a72385 None admin] PATH_INFO: `/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:25.033100 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643612d-3a0e-4cea-8a92-150c76a72385 None admin] RBAC: Authorizing `identity:get_group(group_id=93dde9ef534d4661a2359bb5d1ea03f5)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:25.035402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643612d-3a0e-4cea-8a92-150c76a72385 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:25.038593 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 172/349] 10.222.0.76 () {56 vars in 1180 bytes} [Thu May 26 17:50:24 2022] GET /identity/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5 => generated 226 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:25.040926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e7334-9f94-4872-a5d9-24135d762010 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:25.041420 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e7334-9f94-4872-a5d9-24135d762010 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:25.041867 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e7334-9f94-4872-a5d9-24135d762010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:25.067880 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-247e7334-9f94-4872-a5d9-24135d762010 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:25.069401 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-247e7334-9f94-4872-a5d9-24135d762010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:25.072549 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc838b8-972f-4944-aee2-daaf26b87aaa None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:25.080904 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:25.122378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a03b48c-f5c0-41a2-81d6-73166fc49e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:25.123062 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03b48c-f5c0-41a2-81d6-73166fc49e38 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:25.123337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03b48c-f5c0-41a2-81d6-73166fc49e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:25.123616 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03b48c-f5c0-41a2-81d6-73166fc49e38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:25.124466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a03b48c-f5c0-41a2-81d6-73166fc49e38 None admin] RBAC: Adding query filter params (group.id=93dde9ef534d4661a2359bb5d1ea03f5, role.id=eb1858a5621543028836223abcb3c0a1, scope.project.id=89f44d18dc9b464faeb1bf0bd03569af) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:25.124781 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a03b48c-f5c0-41a2-81d6-73166fc49e38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:25.126279 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a03b48c-f5c0-41a2-81d6-73166fc49e38 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:25.133205 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 178/350] 10.222.0.76 () {56 vars in 1376 bytes} [Thu May 26 17:50:24 2022] GET /identity/v3/role_assignments?group.id=93dde9ef534d4661a2359bb5d1ea03f5&scope.project.id=89f44d18dc9b464faeb1bf0bd03569af&role.id=eb1858a5621543028836223abcb3c0a1 => generated 260 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:25.134714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070d160-67f0-4e84-9931-1d38cb3104e6 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:25.135157 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070d160-67f0-4e84-9931-1d38cb3104e6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:25.135506 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070d160-67f0-4e84-9931-1d38cb3104e6 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:25.137167 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 179/351] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:25.137835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070d160-67f0-4e84-9931-1d38cb3104e6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:25.186885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 173/352] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:25.188528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bb170-66cc-4efa-b5e6-f1a25b2ef629 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:25.188664 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bb170-66cc-4efa-b5e6-f1a25b2ef629 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:25.188920 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bb170-66cc-4efa-b5e6-f1a25b2ef629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:25.210064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-7a8bb170-66cc-4efa-b5e6-f1a25b2ef629 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:25.211454 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-7a8bb170-66cc-4efa-b5e6-f1a25b2ef629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:25.287341 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070d160-67f0-4e84-9931-1d38cb3104e6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:25.299502 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 174/353] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:25.300635 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8bb170-66cc-4efa-b5e6-f1a25b2ef629 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:25.327934 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:25.356897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde7752-f53e-4359-8b12-fb76fff6fb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:25.358116 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde7752-f53e-4359-8b12-fb76fff6fb9c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:25.358665 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde7752-f53e-4359-8b12-fb76fff6fb9c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:25.359118 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde7752-f53e-4359-8b12-fb76fff6fb9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:25.360874 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde7752-f53e-4359-8b12-fb76fff6fb9c None admin] RBAC: Adding query filter params (group.id=93dde9ef534d4661a2359bb5d1ea03f5, role.id=e731efeb1d3c407fb58cf64a02d95796, scope.project.id=89f44d18dc9b464faeb1bf0bd03569af) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:25.361520 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde7752-f53e-4359-8b12-fb76fff6fb9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:25.364602 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde7752-f53e-4359-8b12-fb76fff6fb9c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:25.376740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 180/354] 10.222.0.76 () {56 vars in 1376 bytes} [Thu May 26 17:50:25 2022] GET /identity/v3/role_assignments?group.id=93dde9ef534d4661a2359bb5d1ea03f5&scope.project.id=89f44d18dc9b464faeb1bf0bd03569af&role.id=e731efeb1d3c407fb58cf64a02d95796 => generated 260 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:25.383220 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae4cbb-08c6-4792-a9df-d82a6d183e7e None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:25.383745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae4cbb-08c6-4792-a9df-d82a6d183e7e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:25.384259 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae4cbb-08c6-4792-a9df-d82a6d183e7e None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:25.390973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 181/355] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:25 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:25.391941 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae4cbb-08c6-4792-a9df-d82a6d183e7e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:25.457786 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8bb170-66cc-4efa-b5e6-f1a25b2ef629 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:25.474899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:25.519391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f3811-af8b-4abd-ae90-91533e6fe652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:25.521123 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f3811-af8b-4abd-ae90-91533e6fe652 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:25.521646 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f3811-af8b-4abd-ae90-91533e6fe652 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:25.522216 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f3811-af8b-4abd-ae90-91533e6fe652 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:25.553025 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966f3811-af8b-4abd-ae90-91533e6fe652 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:25.566926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-966f3811-af8b-4abd-ae90-91533e6fe652 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:50:25.566926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:25.566926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:25.566926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:25.566926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:25.566926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:25.566926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:25.566926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:25.566926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:25.566926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:25.566926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:25.566926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:25.566926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:25.566926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:25.566926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:25.566926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:25.567909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:25.567909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:25.567909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:25.567909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:25.567909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:25.567909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:25.567909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:25.567909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:25.567909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:25.567909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:25.567909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:25.567909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:25.567909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:25.567909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:25.567909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:25.567909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:25.568724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:25.568724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:25.568724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:25.568724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:25.568724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:25.568724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:25.568724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:25.568724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:25.568724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:25.568724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:25.568724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:50:25.568724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:25.576869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 175/356] 10.222.0.76 () {56 vars in 1114 bytes} [Thu May 26 17:50:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:25.578790 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f922bf9-d169-45cc-b152-07877671b3c2 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:25.579360 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f922bf9-d169-45cc-b152-07877671b3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:25.579843 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f922bf9-d169-45cc-b152-07877671b3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:25.597961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae4cbb-08c6-4792-a9df-d82a6d183e7e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:25.601538 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-5f922bf9-d169-45cc-b152-07877671b3c2 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:25.603403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-5f922bf9-d169-45cc-b152-07877671b3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:25.654603 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58becb2-d045-434b-bbc8-7758a2c507b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:25.655989 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58becb2-d045-434b-bbc8-7758a2c507b1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:25.656480 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58becb2-d045-434b-bbc8-7758a2c507b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:25.656969 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58becb2-d045-434b-bbc8-7758a2c507b1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:25.674312 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58becb2-d045-434b-bbc8-7758a2c507b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:25.682887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c58becb2-d045-434b-bbc8-7758a2c507b1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:50:25.682887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:25.682887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:25.682887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:25.682887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:25.682887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:25.682887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:25.682887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:25.682887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:25.682887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:25.682887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:25.682887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:25.682887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:25.682887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:25.682887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:25.682887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:25.683933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:25.683933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:25.683933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:25.683933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:25.683933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:25.683933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:25.683933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:25.683933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:25.683933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:25.683933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:25.683933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:25.683933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:25.683933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:25.683933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:25.683933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:25.683933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:25.684931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:25.684931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:25.684931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:25.684931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:25.684931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:25.684931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:25.684931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:25.684931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:25.684931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:25.684931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:25.684931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 17:50:25.684931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:25.684931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 182/357] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:25.685822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e86fa-a849-4746-981e-72bd1a9a256a None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:25.685822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e86fa-a849-4746-981e-72bd1a9a256a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:25.686061 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e86fa-a849-4746-981e-72bd1a9a256a None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:25.687601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 183/358] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:25.688355 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e86fa-a849-4746-981e-72bd1a9a256a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:25.689583 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:25.700156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 176/359] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:25.701932 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01cb47-732c-4151-839f-b5baac71a20b None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:25.702430 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01cb47-732c-4151-839f-b5baac71a20b None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:25.703188 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01cb47-732c-4151-839f-b5baac71a20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:25.722090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-4d01cb47-732c-4151-839f-b5baac71a20b None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:25.724390 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-4d01cb47-732c-4151-839f-b5baac71a20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:25.814732 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 177/360] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:25.816058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01cb47-732c-4151-839f-b5baac71a20b None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:25.839116 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:25.854740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e86fa-a849-4746-981e-72bd1a9a256a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:25.897691 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae31c86-019f-4cbf-a13a-db7edc994cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:25.898424 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae31c86-019f-4cbf-a13a-db7edc994cad None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:25.898712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae31c86-019f-4cbf-a13a-db7edc994cad None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:25.902645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae31c86-019f-4cbf-a13a-db7edc994cad None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:25.903481 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae31c86-019f-4cbf-a13a-db7edc994cad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:25.903804 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae31c86-019f-4cbf-a13a-db7edc994cad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:25.905263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae31c86-019f-4cbf-a13a-db7edc994cad None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:25.921363 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 184/361] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:50:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:25.921363 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae31c86-019f-4cbf-a13a-db7edc994cad None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:25.956203 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01cb47-732c-4151-839f-b5baac71a20b None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:25.998053 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:26.010688 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c484ba5-90bd-4562-9dbd-d1ca61574f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:26.011400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c484ba5-90bd-4562-9dbd-d1ca61574f58 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:26.011694 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c484ba5-90bd-4562-9dbd-d1ca61574f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:26.011979 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c484ba5-90bd-4562-9dbd-d1ca61574f58 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:26.012769 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c484ba5-90bd-4562-9dbd-d1ca61574f58 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:26.013324 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c484ba5-90bd-4562-9dbd-d1ca61574f58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:26.014895 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c484ba5-90bd-4562-9dbd-d1ca61574f58 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:26.027084 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 178/362] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:50:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:26.028019 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c484ba5-90bd-4562-9dbd-d1ca61574f58 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:26.066398 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae31c86-019f-4cbf-a13a-db7edc994cad None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:26.107873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a907d1-fbb3-4f27-9517-571e5e3b2d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:26.109072 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a907d1-fbb3-4f27-9517-571e5e3b2d4e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:26.109556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a907d1-fbb3-4f27-9517-571e5e3b2d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:26.110024 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a907d1-fbb3-4f27-9517-571e5e3b2d4e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:26.123603 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a907d1-fbb3-4f27-9517-571e5e3b2d4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:26.133661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-06a907d1-fbb3-4f27-9517-571e5e3b2d4e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:26.133661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:26.133661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:26.133661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:26.133661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:26.133661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:26.133661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:26.133661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:26.133661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:26.133661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:26.133661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:26.133661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:26.133661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:26.133661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:26.133661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:26.133661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:26.134662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:26.134662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:26.134662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:26.134662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:26.134662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:26.134662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:26.134662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:26.134662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:26.134662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:26.134662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:26.134662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:26.134662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:26.134662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:26.134662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:26.134662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:26.134662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:26.135608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:26.135608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:26.135608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:26.135608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:26.135608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:26.135608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:26.135608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:26.135608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:26.135608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:26.135608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:26.135608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:26.135608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:26.138334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 185/363] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:26.139816 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a907d1-fbb3-4f27-9517-571e5e3b2d4e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:26.150520 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c484ba5-90bd-4562-9dbd-d1ca61574f58 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:26.163042 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:26.193187 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35d635-4056-4cae-aa17-3bec97f2f676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:26.194306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35d635-4056-4cae-aa17-3bec97f2f676 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:26.194742 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35d635-4056-4cae-aa17-3bec97f2f676 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:26.195168 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35d635-4056-4cae-aa17-3bec97f2f676 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:26.209612 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c35d635-4056-4cae-aa17-3bec97f2f676 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:26.218294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-6c35d635-4056-4cae-aa17-3bec97f2f676 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:50:26.218294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:26.218294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:26.218294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:26.218294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:26.218294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:26.218294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:26.218294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:26.218294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:26.218294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:26.218294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:26.218294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:26.218294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:26.218294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:26.218294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:26.218294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:26.219277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:26.219277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:26.219277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:26.219277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:26.219277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:26.219277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:26.219277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:26.219277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:26.219277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:26.219277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:26.219277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:26.219277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:26.219277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:26.219277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:26.219277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:26.219277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:26.220104 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:26.220104 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:26.220104 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:26.220104 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:26.220104 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:26.220104 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:26.220104 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:26.220104 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:26.220104 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:26.220104 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:26.220104 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:50:26.220104 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:26.222892 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 179/364] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:50:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:26.224301 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35d635-4056-4cae-aa17-3bec97f2f676 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:26.275338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a907d1-fbb3-4f27-9517-571e5e3b2d4e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:26.284145 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:26.327523 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eec877-fde0-4d99-968c-1aed0f0e0a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:26.328089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eec877-fde0-4d99-968c-1aed0f0e0a9b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:26.328371 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eec877-fde0-4d99-968c-1aed0f0e0a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:26.328645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eec877-fde0-4d99-968c-1aed0f0e0a9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:26.365395 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35d635-4056-4cae-aa17-3bec97f2f676 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:26.407844 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe839a1-a925-4473-a50f-a8f9a2bb7058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:26.409201 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe839a1-a925-4473-a50f-a8f9a2bb7058 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:26.409800 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe839a1-a925-4473-a50f-a8f9a2bb7058 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:26.410393 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe839a1-a925-4473-a50f-a8f9a2bb7058 None admin] PATH_INFO: `/v3/users/dc5a4de18f074fcea3c0e4eb4fd0eaef` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:26.412525 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe839a1-a925-4473-a50f-a8f9a2bb7058 None admin] RBAC: Authorizing `identity:get_user(user_id=dc5a4de18f074fcea3c0e4eb4fd0eaef)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:26.414621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe839a1-a925-4473-a50f-a8f9a2bb7058 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:26.416851 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 180/365] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:50:26 2022] GET /identity/v3/users/dc5a4de18f074fcea3c0e4eb4fd0eaef => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:26.418130 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe839a1-a925-4473-a50f-a8f9a2bb7058 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:26.454616 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eec877-fde0-4d99-968c-1aed0f0e0a9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:26.458090 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eec877-fde0-4d99-968c-1aed0f0e0a9b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:26.490607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:26.551594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe839a1-a925-4473-a50f-a8f9a2bb7058 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:26.572053 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:26.595916 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6a64e-db6a-4867-82ba-3c99c8bae1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:26.596668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6a64e-db6a-4867-82ba-3c99c8bae1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:26.597167 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6a64e-db6a-4867-82ba-3c99c8bae1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:26.598173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6a64e-db6a-4867-82ba-3c99c8bae1e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:26.598952 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6a64e-db6a-4867-82ba-3c99c8bae1e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:26.599367 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6a64e-db6a-4867-82ba-3c99c8bae1e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:26.600974 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6a64e-db6a-4867-82ba-3c99c8bae1e9 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:26.612155 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 181/366] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:50:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:26.613371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6a64e-db6a-4867-82ba-3c99c8bae1e9 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:26.641366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 186/367] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:50:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:50:26.643017 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eec877-fde0-4d99-968c-1aed0f0e0a9b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:26.744944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6a64e-db6a-4867-82ba-3c99c8bae1e9 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:26.785086 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991f132-c9d0-461e-a353-636a44a1cd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:26.786084 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991f132-c9d0-461e-a353-636a44a1cd50 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:26.786747 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991f132-c9d0-461e-a353-636a44a1cd50 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:26.787231 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991f132-c9d0-461e-a353-636a44a1cd50 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:26.788114 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d991f132-c9d0-461e-a353-636a44a1cd50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:26.788519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d991f132-c9d0-461e-a353-636a44a1cd50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:26.790253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d991f132-c9d0-461e-a353-636a44a1cd50 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:26.795981 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eec877-fde0-4d99-968c-1aed0f0e0a9b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:26.800482 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 182/368] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:26.801875 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991f132-c9d0-461e-a353-636a44a1cd50 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:26.816578 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:26.839274 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96244c66-d273-4c3f-8b17-7071f285189e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:26.840803 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96244c66-d273-4c3f-8b17-7071f285189e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:26.841266 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96244c66-d273-4c3f-8b17-7071f285189e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:26.841731 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96244c66-d273-4c3f-8b17-7071f285189e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:26.842868 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96244c66-d273-4c3f-8b17-7071f285189e None admin] RBAC: Adding query filter params (role.id=5b3eae4902f44cc9a56bf2ec2a8e13f5, user.id=dc5a4de18f074fcea3c0e4eb4fd0eaef, scope.system=all) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:26.843381 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96244c66-d273-4c3f-8b17-7071f285189e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:26.845049 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96244c66-d273-4c3f-8b17-7071f285189e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:26.852704 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:26.859995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 187/369] 10.222.0.76 () {56 vars in 1308 bytes} [Thu May 26 17:50:26 2022] GET /identity/v3/role_assignments?user.id=dc5a4de18f074fcea3c0e4eb4fd0eaef&scope.system=all&role.id=5b3eae4902f44cc9a56bf2ec2a8e13f5 => generated 510 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:26.861204 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96244c66-d273-4c3f-8b17-7071f285189e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:26.929048 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991f132-c9d0-461e-a353-636a44a1cd50 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:26.974462 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ac9ff-188b-47ce-a55a-278d9f58e729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:26.975422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ac9ff-188b-47ce-a55a-278d9f58e729 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:26.975575 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ac9ff-188b-47ce-a55a-278d9f58e729 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:26.975904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ac9ff-188b-47ce-a55a-278d9f58e729 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:27.074225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ac9ff-188b-47ce-a55a-278d9f58e729 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:27.074225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ac9ff-188b-47ce-a55a-278d9f58e729 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:27.086271 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96244c66-d273-4c3f-8b17-7071f285189e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:27.129582 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:27.169293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8440bc-ab28-40f3-9411-6f4f68dc1a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:27.170992 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8440bc-ab28-40f3-9411-6f4f68dc1a39 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:27.171505 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8440bc-ab28-40f3-9411-6f4f68dc1a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:27.172015 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8440bc-ab28-40f3-9411-6f4f68dc1a39 None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:27.174488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8440bc-ab28-40f3-9411-6f4f68dc1a39 None admin] RBAC: Authorizing `identity:get_project(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:27.177880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8440bc-ab28-40f3-9411-6f4f68dc1a39 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:27.181899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 188/370] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:26 2022] GET /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94 => generated 302 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:27.183550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8440bc-ab28-40f3-9411-6f4f68dc1a39 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:27.274303 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:27.290407 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 183/371] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:50:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:50:27.291512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ac9ff-188b-47ce-a55a-278d9f58e729 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:27.381862 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8440bc-ab28-40f3-9411-6f4f68dc1a39 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:27.441794 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009a222-4420-4e39-a2cf-56409cbd13c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:27.442861 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009a222-4420-4e39-a2cf-56409cbd13c0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:27.443309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009a222-4420-4e39-a2cf-56409cbd13c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:27.443783 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009a222-4420-4e39-a2cf-56409cbd13c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:27.463194 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ac9ff-188b-47ce-a55a-278d9f58e729 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:27.509643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91802170-a786-405b-9556-5fbc37a784bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:27.511887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91802170-a786-405b-9556-5fbc37a784bb None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:27.512315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91802170-a786-405b-9556-5fbc37a784bb None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:27.512627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91802170-a786-405b-9556-5fbc37a784bb None admin] PATH_INFO: `/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:27.514382 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91802170-a786-405b-9556-5fbc37a784bb None admin] RBAC: Authorizing `identity:get_group(group_id=93dde9ef534d4661a2359bb5d1ea03f5)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:27.516101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91802170-a786-405b-9556-5fbc37a784bb None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:27.518165 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 184/372] 10.222.0.76 () {56 vars in 1180 bytes} [Thu May 26 17:50:27 2022] GET /identity/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:27.523414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91802170-a786-405b-9556-5fbc37a784bb None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:27.541305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:27.583957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009a222-4420-4e39-a2cf-56409cbd13c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:27.587083 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009a222-4420-4e39-a2cf-56409cbd13c0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:27.683546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:27.698908 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91802170-a786-405b-9556-5fbc37a784bb None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:27.743999 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618b274-6e1b-4d98-8b15-3e97d80107f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:27.744948 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618b274-6e1b-4d98-8b15-3e97d80107f9 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:27.745328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618b274-6e1b-4d98-8b15-3e97d80107f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:27.745694 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618b274-6e1b-4d98-8b15-3e97d80107f9 None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:27.747260 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618b274-6e1b-4d98-8b15-3e97d80107f9 None admin] RBAC: Authorizing `identity:get_project(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:27.749056 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618b274-6e1b-4d98-8b15-3e97d80107f9 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:27.751616 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 185/373] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:27 2022] GET /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:27.752563 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618b274-6e1b-4d98-8b15-3e97d80107f9 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:27.801478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 189/374] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:50:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:50:27.803024 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009a222-4420-4e39-a2cf-56409cbd13c0 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:27.859172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:27.901877 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618b274-6e1b-4d98-8b15-3e97d80107f9 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:27.947480 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8475d-7a35-4c2c-8059-90871365b31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:27.948063 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8475d-7a35-4c2c-8059-90871365b31b None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:27.948354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8475d-7a35-4c2c-8059-90871365b31b None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:27.948631 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8475d-7a35-4c2c-8059-90871365b31b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:27.949498 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8475d-7a35-4c2c-8059-90871365b31b None admin] RBAC: Adding query filter params (group.id=93dde9ef534d4661a2359bb5d1ea03f5, role.id=e731efeb1d3c407fb58cf64a02d95796, scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:27.949839 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8475d-7a35-4c2c-8059-90871365b31b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:27.951313 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8475d-7a35-4c2c-8059-90871365b31b None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:27.963683 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 186/375] 10.222.0.76 () {56 vars in 1376 bytes} [Thu May 26 17:50:27 2022] GET /identity/v3/role_assignments?group.id=93dde9ef534d4661a2359bb5d1ea03f5&scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94&role.id=e731efeb1d3c407fb58cf64a02d95796 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:27.964720 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8475d-7a35-4c2c-8059-90871365b31b None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:28.038363 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009a222-4420-4e39-a2cf-56409cbd13c0 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:28.126172 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d1edc8-3507-48a0-a617-193b562f0f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:28.127258 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1edc8-3507-48a0-a617-193b562f0f35 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:28.127581 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1edc8-3507-48a0-a617-193b562f0f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:28.127957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1edc8-3507-48a0-a617-193b562f0f35 None admin] PATH_INFO: `/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:28.136052 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d1edc8-3507-48a0-a617-193b562f0f35 None admin] RBAC: Authorizing `identity:get_group(group_id=93dde9ef534d4661a2359bb5d1ea03f5)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:28.137871 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d1edc8-3507-48a0-a617-193b562f0f35 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:28.148172 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 190/376] 10.222.0.76 () {56 vars in 1180 bytes} [Thu May 26 17:50:27 2022] GET /identity/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5 => generated 226 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:28.150325 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8599c391-4146-47c4-931e-a66b175f161e None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:28.150931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8599c391-4146-47c4-931e-a66b175f161e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:28.151735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8599c391-4146-47c4-931e-a66b175f161e None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:28.154043 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 191/377] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:28.157393 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8599c391-4146-47c4-931e-a66b175f161e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:28.165631 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8475d-7a35-4c2c-8059-90871365b31b None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:28.178096 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:28.224715 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acbb7ef-5a38-45aa-a603-e82519b5a502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:28.226389 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbb7ef-5a38-45aa-a603-e82519b5a502 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:28.227105 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbb7ef-5a38-45aa-a603-e82519b5a502 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:28.227632 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbb7ef-5a38-45aa-a603-e82519b5a502 None admin] PATH_INFO: `/v3/projects/5a106ee939504b208f8d5ca1deb61f2d` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:28.230046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acbb7ef-5a38-45aa-a603-e82519b5a502 None admin] RBAC: Authorizing `identity:get_project(project_id=5a106ee939504b208f8d5ca1deb61f2d)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:28.232788 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acbb7ef-5a38-45aa-a603-e82519b5a502 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:28.235878 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 187/378] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:27 2022] GET /identity/v3/projects/5a106ee939504b208f8d5ca1deb61f2d => generated 344 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:28.238459 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e1f1e-6971-434e-beda-d044d1be15fe None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:28.239141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e1f1e-6971-434e-beda-d044d1be15fe None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:28.239741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e1f1e-6971-434e-beda-d044d1be15fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:28.263462 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-b32e1f1e-6971-434e-beda-d044d1be15fe None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:28.265796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-b32e1f1e-6971-434e-beda-d044d1be15fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:28.269055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:28.301784 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8599c391-4146-47c4-931e-a66b175f161e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:28.344317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b01d4c-7bc9-414f-814a-d484fe916d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:28.345048 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b01d4c-7bc9-414f-814a-d484fe916d44 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:28.345341 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b01d4c-7bc9-414f-814a-d484fe916d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:28.345625 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b01d4c-7bc9-414f-814a-d484fe916d44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:28.346532 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b01d4c-7bc9-414f-814a-d484fe916d44 None admin] RBAC: Adding query filter params (group.id=93dde9ef534d4661a2359bb5d1ea03f5, role.id=eb1858a5621543028836223abcb3c0a1, scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:28.346872 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b01d4c-7bc9-414f-814a-d484fe916d44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:28.349299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b01d4c-7bc9-414f-814a-d484fe916d44 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:28.356290 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 192/379] 10.222.0.76 () {56 vars in 1376 bytes} [Thu May 26 17:50:28 2022] GET /identity/v3/role_assignments?group.id=93dde9ef534d4661a2359bb5d1ea03f5&scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94&role.id=eb1858a5621543028836223abcb3c0a1 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:28.357205 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b01d4c-7bc9-414f-814a-d484fe916d44 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:28.369699 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 188/380] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:28.415885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e1f1e-6971-434e-beda-d044d1be15fe None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:28.458233 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:28.494225 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b01d4c-7bc9-414f-814a-d484fe916d44 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:28.533574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a026417-7954-4c85-9448-7a1c74623732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:28.534520 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a026417-7954-4c85-9448-7a1c74623732 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:28.535581 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a026417-7954-4c85-9448-7a1c74623732 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:28.535581 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a026417-7954-4c85-9448-7a1c74623732 None admin] PATH_INFO: `/v3/groups/0edbee58926141c2b39b948a6cfe3984` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:28.542484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a026417-7954-4c85-9448-7a1c74623732 None admin] RBAC: Authorizing `identity:get_group(group_id=0edbee58926141c2b39b948a6cfe3984)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:28.544198 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a026417-7954-4c85-9448-7a1c74623732 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:28.546384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 193/381] 10.222.0.76 () {56 vars in 1180 bytes} [Thu May 26 17:50:28 2022] GET /identity/v3/groups/0edbee58926141c2b39b948a6cfe3984 => generated 229 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:28.554537 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a026417-7954-4c85-9448-7a1c74623732 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:28.654181 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e1f1e-6971-434e-beda-d044d1be15fe None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:28.721570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a026417-7954-4c85-9448-7a1c74623732 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:28.724571 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e7287-ea8e-4aba-9a9e-853d937435f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:28.726833 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e7287-ea8e-4aba-9a9e-853d937435f0 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:28.727307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e7287-ea8e-4aba-9a9e-853d937435f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:28.727677 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e7287-ea8e-4aba-9a9e-853d937435f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:28.741281 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:28.749877 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e7287-ea8e-4aba-9a9e-853d937435f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:28.766001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-781e7287-ea8e-4aba-9a9e-853d937435f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:28.766001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:28.766001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:28.766001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:28.766001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:28.766001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:28.766001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:28.766001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:28.766001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:28.766001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:28.766001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:28.766001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:28.766001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:28.766001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:28.766001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:28.766001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:28.767029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:28.767029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:28.767029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:28.767029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:28.767029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:28.767029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:28.767029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:28.767029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:28.767029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:28.767029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:28.767029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:28.767029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:28.767029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:28.767029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:28.767029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:28.767029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:28.767817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:28.767817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:28.767817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:28.767817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:28.767817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:28.767817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:28.767817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:28.767817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:28.767817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:28.767817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:28.767817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:28.767817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:28.767817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 189/382] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:50:28.768561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea217da-1078-4258-87e1-dc1744768520 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:28.769701 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea217da-1078-4258-87e1-dc1744768520 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:28.769701 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea217da-1078-4258-87e1-dc1744768520 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:28.771927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013670a6-3c29-4a7e-9dbf-24daff33ba18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:28.772086 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 190/383] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:28.772180 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013670a6-3c29-4a7e-9dbf-24daff33ba18 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:28.772524 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013670a6-3c29-4a7e-9dbf-24daff33ba18 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:28.773026 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013670a6-3c29-4a7e-9dbf-24daff33ba18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:28.778237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013670a6-3c29-4a7e-9dbf-24daff33ba18 None admin] RBAC: Adding query filter params (group.id=0edbee58926141c2b39b948a6cfe3984, role.id=4fa8f06085c741a5b3cf64e63cc0754f, scope.project.id=5a106ee939504b208f8d5ca1deb61f2d) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:28.778237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013670a6-3c29-4a7e-9dbf-24daff33ba18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:28.779532 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013670a6-3c29-4a7e-9dbf-24daff33ba18 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:28.779797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa21d2-a95c-4915-bebd-02bc9cc81ea7 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:28.780094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa21d2-a95c-4915-bebd-02bc9cc81ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:28.780380 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa21d2-a95c-4915-bebd-02bc9cc81ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:28.793663 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 194/384] 10.222.0.76 () {56 vars in 1376 bytes} [Thu May 26 17:50:28 2022] GET /identity/v3/role_assignments?group.id=0edbee58926141c2b39b948a6cfe3984&scope.project.id=5a106ee939504b208f8d5ca1deb61f2d&role.id=4fa8f06085c741a5b3cf64e63cc0754f => generated 260 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:28.794772 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013670a6-3c29-4a7e-9dbf-24daff33ba18 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:28.805571 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-19aa21d2-a95c-4915-bebd-02bc9cc81ea7 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:28.810162 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-19aa21d2-a95c-4915-bebd-02bc9cc81ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:28.814518 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:28.964097 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013670a6-3c29-4a7e-9dbf-24daff33ba18 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:28.974117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 191/385] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:29.002428 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa21d2-a95c-4915-bebd-02bc9cc81ea7 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:29.008100 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba3c27-7348-449e-89c1-c51160e485c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:29.008798 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba3c27-7348-449e-89c1-c51160e485c3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:29.009082 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba3c27-7348-449e-89c1-c51160e485c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:29.009361 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba3c27-7348-449e-89c1-c51160e485c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:29.010127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba3c27-7348-449e-89c1-c51160e485c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:29.010473 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba3c27-7348-449e-89c1-c51160e485c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:29.011993 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba3c27-7348-449e-89c1-c51160e485c3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:29.024863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 195/386] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:50:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:29.030372 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba3c27-7348-449e-89c1-c51160e485c3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:29.118385 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:29.170677 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba3c27-7348-449e-89c1-c51160e485c3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:29.220807 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa21d2-a95c-4915-bebd-02bc9cc81ea7 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:29.235478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110847d-80a9-4142-be48-68c178230e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:29.236534 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110847d-80a9-4142-be48-68c178230e16 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:29.236928 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110847d-80a9-4142-be48-68c178230e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:29.237336 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110847d-80a9-4142-be48-68c178230e16 None admin] PATH_INFO: `/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:29.239086 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110847d-80a9-4142-be48-68c178230e16 None admin] RBAC: Authorizing `identity:get_group(group_id=93dde9ef534d4661a2359bb5d1ea03f5)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:29.241051 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110847d-80a9-4142-be48-68c178230e16 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:29.244062 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 196/387] 10.222.0.76 () {56 vars in 1180 bytes} [Thu May 26 17:50:29 2022] GET /identity/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:29.251681 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110847d-80a9-4142-be48-68c178230e16 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:29.288253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:29.327618 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefafaa-3034-4603-9954-4c37c0513ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:29.328695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefafaa-3034-4603-9954-4c37c0513ffd None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:29.329131 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefafaa-3034-4603-9954-4c37c0513ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:29.329579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefafaa-3034-4603-9954-4c37c0513ffd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:29.367579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefafaa-3034-4603-9954-4c37c0513ffd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:29.387723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-0cefafaa-3034-4603-9954-4c37c0513ffd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:50:29.387723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:29.387723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:29.387723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:29.387723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:29.387723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:29.387723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:29.387723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:29.387723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:29.387723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:29.387723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:29.387723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:29.387723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:29.387723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:29.387723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:29.387723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:29.389371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:29.389371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:29.389371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:29.389371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:29.389371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:29.389371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:29.389371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:29.389371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:29.389371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:29.389371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:29.389371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:29.389371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:29.389371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:29.389371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:29.389371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:29.389371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:29.390697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:29.390697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:29.390697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:29.390697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:29.390697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:29.390697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:29.390697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:29.390697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:29.390697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:29.390697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:29.390697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:50:29.390697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:29.393312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 192/388] 10.222.0.76 () {56 vars in 1114 bytes} [Thu May 26 17:50:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:29.403315 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110847d-80a9-4142-be48-68c178230e16 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:29.403869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefafaa-3034-4603-9954-4c37c0513ffd None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:29.450847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13876128-707f-4a43-9fc6-db5bfb560767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:29.451881 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13876128-707f-4a43-9fc6-db5bfb560767 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:29.452274 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13876128-707f-4a43-9fc6-db5bfb560767 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:29.452680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13876128-707f-4a43-9fc6-db5bfb560767 None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:29.454039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13876128-707f-4a43-9fc6-db5bfb560767 None admin] RBAC: Authorizing `identity:get_project(project_id=89f44d18dc9b464faeb1bf0bd03569af)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:29.458216 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13876128-707f-4a43-9fc6-db5bfb560767 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:29.458529 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:29.458897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 197/389] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:29 2022] GET /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:29.471930 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13876128-707f-4a43-9fc6-db5bfb560767 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:29.709572 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13876128-707f-4a43-9fc6-db5bfb560767 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:29.742071 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefafaa-3034-4603-9954-4c37c0513ffd None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:29.788566 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ec2db-a4bc-49c0-a220-d4fd3b9eeaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:29.792066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ec2db-a4bc-49c0-a220-d4fd3b9eeaef None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:29.792658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ec2db-a4bc-49c0-a220-d4fd3b9eeaef None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:29.793177 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ec2db-a4bc-49c0-a220-d4fd3b9eeaef None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af/groups/93dde9ef534d4661a2359bb5d1ea03f5/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:29.800325 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ec2db-a4bc-49c0-a220-d4fd3b9eeaef None admin] RBAC: Authorizing `identity:create_grant(project_id=89f44d18dc9b464faeb1bf0bd03569af, group_id=93dde9ef534d4661a2359bb5d1ea03f5, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:29.805253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ec2db-a4bc-49c0-a220-d4fd3b9eeaef None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:29.845702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 198/390] 10.222.0.76 () {58 vars in 1442 bytes} [Thu May 26 17:50:29 2022] PUT /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af/groups/93dde9ef534d4661a2359bb5d1ea03f5/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:50:29.862750 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933d243-016c-4f02-a8c0-54b59a79bb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:29.863903 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933d243-016c-4f02-a8c0-54b59a79bb43 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:29.864424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933d243-016c-4f02-a8c0-54b59a79bb43 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:29.864930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933d243-016c-4f02-a8c0-54b59a79bb43 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:29.866460 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933d243-016c-4f02-a8c0-54b59a79bb43 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:29.867040 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933d243-016c-4f02-a8c0-54b59a79bb43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:29.869728 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933d243-016c-4f02-a8c0-54b59a79bb43 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:29.897363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 193/391] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:50:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 494 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:29.908020 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ec2db-a4bc-49c0-a220-d4fd3b9eeaef None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:30.044937 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:30.121832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ec2db-a4bc-49c0-a220-d4fd3b9eeaef None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:30.187956 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34816b41-71a4-406d-8f6b-0a45e7c72d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:30.188852 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34816b41-71a4-406d-8f6b-0a45e7c72d57 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:30.189214 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34816b41-71a4-406d-8f6b-0a45e7c72d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:30.189492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34816b41-71a4-406d-8f6b-0a45e7c72d57 None admin] PATH_INFO: `/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:30.195292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34816b41-71a4-406d-8f6b-0a45e7c72d57 None admin] RBAC: Authorizing `identity:get_group(group_id=93dde9ef534d4661a2359bb5d1ea03f5)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:30.196931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34816b41-71a4-406d-8f6b-0a45e7c72d57 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:30.199207 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 199/392] 10.222.0.76 () {56 vars in 1180 bytes} [Thu May 26 17:50:29 2022] GET /identity/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5 => generated 226 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:30.208226 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933d243-016c-4f02-a8c0-54b59a79bb43 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:30.282729 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:30.361758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933d243-016c-4f02-a8c0-54b59a79bb43 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:30.405523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad32a90-e07b-4416-9237-995ea0b219e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:30.406520 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad32a90-e07b-4416-9237-995ea0b219e6 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:30.406810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad32a90-e07b-4416-9237-995ea0b219e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:30.407092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad32a90-e07b-4416-9237-995ea0b219e6 None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:30.409154 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad32a90-e07b-4416-9237-995ea0b219e6 None admin] RBAC: Authorizing `identity:get_project(project_id=89f44d18dc9b464faeb1bf0bd03569af)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:30.410901 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad32a90-e07b-4416-9237-995ea0b219e6 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:30.413271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 194/393] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:30 2022] GET /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:30.421113 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34816b41-71a4-406d-8f6b-0a45e7c72d57 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:30.577721 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34816b41-71a4-406d-8f6b-0a45e7c72d57 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:30.638288 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:30.656576 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f4c23-ed1b-4eaa-a150-c9be07ca38dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:30.657623 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f4c23-ed1b-4eaa-a150-c9be07ca38dc None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:30.657956 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f4c23-ed1b-4eaa-a150-c9be07ca38dc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:30.658330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f4c23-ed1b-4eaa-a150-c9be07ca38dc None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af/groups/93dde9ef534d4661a2359bb5d1ea03f5/roles/e731efeb1d3c407fb58cf64a02d95796` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:30.661613 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f4c23-ed1b-4eaa-a150-c9be07ca38dc None admin] RBAC: Authorizing `identity:create_grant(project_id=89f44d18dc9b464faeb1bf0bd03569af, group_id=93dde9ef534d4661a2359bb5d1ea03f5, role_id=e731efeb1d3c407fb58cf64a02d95796)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:30.663239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f4c23-ed1b-4eaa-a150-c9be07ca38dc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:30.683375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 200/394] 10.222.0.76 () {58 vars in 1442 bytes} [Thu May 26 17:50:30 2022] PUT /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af/groups/93dde9ef534d4661a2359bb5d1ea03f5/roles/e731efeb1d3c407fb58cf64a02d95796 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:50:31.435781 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be4c63-e70e-4b49-a9a8-5916369f959a None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:31.436306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be4c63-e70e-4b49-a9a8-5916369f959a None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:31.436306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be4c63-e70e-4b49-a9a8-5916369f959a None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:31.437992 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 195/395] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:31.443882 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080d183-985b-44a6-8df0-29ef3bc29337 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:31.444391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080d183-985b-44a6-8df0-29ef3bc29337 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:31.444540 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080d183-985b-44a6-8df0-29ef3bc29337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:31.478520 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d080d183-985b-44a6-8df0-29ef3bc29337 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:31.480120 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d080d183-985b-44a6-8df0-29ef3bc29337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:31.592965 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 201/396] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:31.627422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be4c63-e70e-4b49-a9a8-5916369f959a None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:31.776734 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be4c63-e70e-4b49-a9a8-5916369f959a None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:31.808975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:31.837062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ea58e-5231-41a5-930e-e09bc5e127bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:31.837896 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ea58e-5231-41a5-930e-e09bc5e127bf None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:31.838184 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ea58e-5231-41a5-930e-e09bc5e127bf None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:31.838492 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ea58e-5231-41a5-930e-e09bc5e127bf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:31.851265 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ea58e-5231-41a5-930e-e09bc5e127bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:31.859630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-691ea58e-5231-41a5-930e-e09bc5e127bf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:50:31.859630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:31.859630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:31.859630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:31.859630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:31.859630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:31.859630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:31.859630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:31.859630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:31.859630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:31.859630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:31.859630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:31.859630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:31.859630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:31.859630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:31.859630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:31.860671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:31.860671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:31.860671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:31.860671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:31.860671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:31.860671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:31.860671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:31.860671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:31.860671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:31.860671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:31.860671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:31.860671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:31.860671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:31.860671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:31.860671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:31.860671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:31.861512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:31.861512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:31.861512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:31.861512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:31.861512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:31.861512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:31.861512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:31.861512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:31.861512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:31.861512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:31.861512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:50:31.861512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:31.861512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 196/397] 10.222.0.76 () {56 vars in 1114 bytes} [Thu May 26 17:50:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:50:31.867179 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080d183-985b-44a6-8df0-29ef3bc29337 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:32.017395 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895e18b-3ae6-4c13-8388-add178063818 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:32.017986 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895e18b-3ae6-4c13-8388-add178063818 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:32.018300 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895e18b-3ae6-4c13-8388-add178063818 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:32.020126 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 197/398] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:32.025767 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d32bbe-f5c1-469e-a121-13806811db6a None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:32.026427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d32bbe-f5c1-469e-a121-13806811db6a None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:32.026962 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d32bbe-f5c1-469e-a121-13806811db6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:32.083537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-18d32bbe-f5c1-469e-a121-13806811db6a None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:32.085364 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-18d32bbe-f5c1-469e-a121-13806811db6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:32.112536 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080d183-985b-44a6-8df0-29ef3bc29337 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:32.154606 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:32.196835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0f57c-9be4-402d-99a7-308183fc2f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:32.198765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0f57c-9be4-402d-99a7-308183fc2f14 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:32.199056 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0f57c-9be4-402d-99a7-308183fc2f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:32.199331 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0f57c-9be4-402d-99a7-308183fc2f14 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:32.200100 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0f57c-9be4-402d-99a7-308183fc2f14 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:32.200410 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0f57c-9be4-402d-99a7-308183fc2f14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:32.201809 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0f57c-9be4-402d-99a7-308183fc2f14 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:32.227407 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 202/399] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:50:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:32.243552 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0f57c-9be4-402d-99a7-308183fc2f14 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:32.245192 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 198/400] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:32.281149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d32bbe-f5c1-469e-a121-13806811db6a None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:32.410408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0f57c-9be4-402d-99a7-308183fc2f14 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:32.415657 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d32bbe-f5c1-469e-a121-13806811db6a None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:32.428974 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:32.462018 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ebba68-4ebf-4255-83be-ab2c021f661e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:32.462672 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7b240-2551-46ab-837f-f18d0dc9e702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:32.463018 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ebba68-4ebf-4255-83be-ab2c021f661e None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:32.463253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ebba68-4ebf-4255-83be-ab2c021f661e None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:32.463431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ebba68-4ebf-4255-83be-ab2c021f661e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:32.463576 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7b240-2551-46ab-837f-f18d0dc9e702 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:32.466215 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7b240-2551-46ab-837f-f18d0dc9e702 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:32.466215 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7b240-2551-46ab-837f-f18d0dc9e702 None admin] PATH_INFO: `/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:32.466215 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7b240-2551-46ab-837f-f18d0dc9e702 None admin] RBAC: Authorizing `identity:get_group(group_id=93dde9ef534d4661a2359bb5d1ea03f5)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:32.467129 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7b240-2551-46ab-837f-f18d0dc9e702 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:32.469230 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 203/401] 10.222.0.76 () {56 vars in 1180 bytes} [Thu May 26 17:50:32 2022] GET /identity/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5 => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:32.470629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352d44f-2aa0-413b-bd93-331bc22381b9 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:32.470915 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352d44f-2aa0-413b-bd93-331bc22381b9 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:32.471187 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352d44f-2aa0-413b-bd93-331bc22381b9 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:32.473961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 204/402] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:32.478983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352d44f-2aa0-413b-bd93-331bc22381b9 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:32.485786 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ebba68-4ebf-4255-83be-ab2c021f661e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:32.495964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-28ebba68-4ebf-4255-83be-ab2c021f661e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:32.495964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:32.495964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:32.495964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:32.495964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:32.495964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:32.495964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:32.495964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:32.495964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:32.495964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:32.495964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:32.495964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:32.495964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:32.495964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:32.495964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:32.495964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:32.496987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:32.496987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:32.496987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:32.496987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:32.496987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:32.496987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:32.496987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:32.496987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:32.496987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:32.496987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:32.496987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:32.496987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:32.496987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:32.496987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:32.496987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:32.496987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:32.497876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:32.497876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:32.497876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:32.497876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:32.497876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:32.497876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:32.497876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:32.497876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:32.497876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:32.497876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:32.497876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:32.497876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:32.498548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 199/403] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:50:32.502423 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf3f13-2946-45bc-9015-23a9d3ed689e None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:32.502423 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf3f13-2946-45bc-9015-23a9d3ed689e None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:32.502423 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf3f13-2946-45bc-9015-23a9d3ed689e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:32.523415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-bebf3f13-2946-45bc-9015-23a9d3ed689e None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:32.524830 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-bebf3f13-2946-45bc-9015-23a9d3ed689e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:32.559782 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:32.610261 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 200/404] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:32.610261 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf3f13-2946-45bc-9015-23a9d3ed689e None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:32.648439 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352d44f-2aa0-413b-bd93-331bc22381b9 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:32.693379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a592e07-31aa-4e99-b7e3-bd124a7f9dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:32.694665 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a592e07-31aa-4e99-b7e3-bd124a7f9dce None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:32.695090 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a592e07-31aa-4e99-b7e3-bd124a7f9dce None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:32.695502 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a592e07-31aa-4e99-b7e3-bd124a7f9dce None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:32.697064 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a592e07-31aa-4e99-b7e3-bd124a7f9dce None admin] RBAC: Authorizing `identity:get_project(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:32.698826 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a592e07-31aa-4e99-b7e3-bd124a7f9dce None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:32.701671 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 205/405] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:32 2022] GET /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:32.702812 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a592e07-31aa-4e99-b7e3-bd124a7f9dce None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:32.731126 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:32.735186 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf3f13-2946-45bc-9015-23a9d3ed689e None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:32.777001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12734597-6c0d-4951-b6de-14fb7f15e5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:32.778223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12734597-6c0d-4951-b6de-14fb7f15e5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:32.778761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12734597-6c0d-4951-b6de-14fb7f15e5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:32.779287 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12734597-6c0d-4951-b6de-14fb7f15e5a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:32.780539 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12734597-6c0d-4951-b6de-14fb7f15e5a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:32.781100 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12734597-6c0d-4951-b6de-14fb7f15e5a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:32.782942 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12734597-6c0d-4951-b6de-14fb7f15e5a3 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:32.799021 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 201/406] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:50:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:32.800441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12734597-6c0d-4951-b6de-14fb7f15e5a3 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:32.852424 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a592e07-31aa-4e99-b7e3-bd124a7f9dce None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:32.872511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:32.895032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e2edaf-cf05-44ea-9960-13484fb091c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:32.896258 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2edaf-cf05-44ea-9960-13484fb091c0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:32.896764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2edaf-cf05-44ea-9960-13484fb091c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:32.897236 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2edaf-cf05-44ea-9960-13484fb091c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:32.911603 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e2edaf-cf05-44ea-9960-13484fb091c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:32.920334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-32e2edaf-cf05-44ea-9960-13484fb091c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:50:32.920334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:32.920334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:32.920334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:32.920334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:32.920334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:32.920334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:32.920334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:32.920334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:32.920334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:32.920334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:32.920334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:32.920334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:32.920334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:32.920334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:32.920334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:32.921306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:32.921306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:32.921306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:32.921306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:32.921306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:32.921306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:32.921306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:32.921306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:32.921306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:32.921306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:32.921306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:32.921306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:32.921306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:32.921306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:32.921306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:32.921306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:32.922152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:32.922152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:32.922152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:32.922152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:32.922152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:32.922152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:32.922152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:32.922152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:32.922152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:32.922152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:32.922152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:50:32.922152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:32.922808 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12734597-6c0d-4951-b6de-14fb7f15e5a3 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:32.925006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 206/407] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:50:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:32.926570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e2edaf-cf05-44ea-9960-13484fb091c0 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:32.961035 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c3293-b828-4067-a104-e1fee3311bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:32.962254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c3293-b828-4067-a104-e1fee3311bc6 None admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:32.962618 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c3293-b828-4067-a104-e1fee3311bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:32.962971 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c3293-b828-4067-a104-e1fee3311bc6 None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94/groups/93dde9ef534d4661a2359bb5d1ea03f5/roles/e731efeb1d3c407fb58cf64a02d95796` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:32.965593 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c3293-b828-4067-a104-e1fee3311bc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94, group_id=93dde9ef534d4661a2359bb5d1ea03f5, role_id=e731efeb1d3c407fb58cf64a02d95796)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:32.967270 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c3293-b828-4067-a104-e1fee3311bc6 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:32.992141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 202/408] 10.222.0.76 () {58 vars in 1442 bytes} [Thu May 26 17:50:32 2022] PUT /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94/groups/93dde9ef534d4661a2359bb5d1ea03f5/roles/e731efeb1d3c407fb58cf64a02d95796 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:50:32.994299 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c3293-b828-4067-a104-e1fee3311bc6 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:33.043078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:33.057170 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e2edaf-cf05-44ea-9960-13484fb091c0 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:33.101146 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10011c0b-aafa-441a-9101-b843d486c7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:33.102887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10011c0b-aafa-441a-9101-b843d486c7dc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:33.103532 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10011c0b-aafa-441a-9101-b843d486c7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:33.104032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10011c0b-aafa-441a-9101-b843d486c7dc None admin] PATH_INFO: `/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:33.105868 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10011c0b-aafa-441a-9101-b843d486c7dc None admin] RBAC: Authorizing `identity:get_group(group_id=93dde9ef534d4661a2359bb5d1ea03f5)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:33.107942 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10011c0b-aafa-441a-9101-b843d486c7dc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:33.110956 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 207/409] 10.222.0.76 () {56 vars in 1180 bytes} [Thu May 26 17:50:32 2022] GET /identity/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:33.122085 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10011c0b-aafa-441a-9101-b843d486c7dc None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:33.226922 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c3293-b828-4067-a104-e1fee3311bc6 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:33.288159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3125c30-05b0-42c7-8016-8c19316dc8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:33.289299 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3125c30-05b0-42c7-8016-8c19316dc8fb None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:33.290165 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3125c30-05b0-42c7-8016-8c19316dc8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:33.291023 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3125c30-05b0-42c7-8016-8c19316dc8fb None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:33.291509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:33.292584 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3125c30-05b0-42c7-8016-8c19316dc8fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:33.292823 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3125c30-05b0-42c7-8016-8c19316dc8fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:33.295051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3125c30-05b0-42c7-8016-8c19316dc8fb None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:33.307960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 203/410] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:33.313210 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3125c30-05b0-42c7-8016-8c19316dc8fb None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:33.368589 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10011c0b-aafa-441a-9101-b843d486c7dc None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:33.438988 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:33.446272 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb6f85-6089-4915-904e-a1bb0b7f913d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:33.447290 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb6f85-6089-4915-904e-a1bb0b7f913d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:33.447611 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb6f85-6089-4915-904e-a1bb0b7f913d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:33.447920 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb6f85-6089-4915-904e-a1bb0b7f913d None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:33.449988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb6f85-6089-4915-904e-a1bb0b7f913d None admin] RBAC: Authorizing `identity:get_project(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:33.452099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb6f85-6089-4915-904e-a1bb0b7f913d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:33.455678 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 208/411] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:33 2022] GET /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94 => generated 302 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:33.457498 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453bdfa-315b-4f88-b82d-b5bf18061dd3 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:33.457892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453bdfa-315b-4f88-b82d-b5bf18061dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:33.458813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453bdfa-315b-4f88-b82d-b5bf18061dd3 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:33.460938 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 209/412] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:33.461968 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453bdfa-315b-4f88-b82d-b5bf18061dd3 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:33.467076 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3125c30-05b0-42c7-8016-8c19316dc8fb None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:33.521603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8bae1-e124-4394-b0a0-b8ee6f0f3ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:33.522805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8bae1-e124-4394-b0a0-b8ee6f0f3ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:33.523227 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8bae1-e124-4394-b0a0-b8ee6f0f3ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:33.523739 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8bae1-e124-4394-b0a0-b8ee6f0f3ab2 None admin] PATH_INFO: `/v3/groups/0edbee58926141c2b39b948a6cfe3984` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:33.525417 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d8bae1-e124-4394-b0a0-b8ee6f0f3ab2 None admin] RBAC: Authorizing `identity:get_group(group_id=0edbee58926141c2b39b948a6cfe3984)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:33.527351 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d8bae1-e124-4394-b0a0-b8ee6f0f3ab2 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:33.530273 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 204/413] 10.222.0.76 () {56 vars in 1180 bytes} [Thu May 26 17:50:33 2022] GET /identity/v3/groups/0edbee58926141c2b39b948a6cfe3984 => generated 229 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:33.532177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab77f73-63a9-4762-bd6b-e45636b7e1c2 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:33.532549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab77f73-63a9-4762-bd6b-e45636b7e1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:33.532934 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab77f73-63a9-4762-bd6b-e45636b7e1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:33.558378 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-aab77f73-63a9-4762-bd6b-e45636b7e1c2 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:33.559795 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-aab77f73-63a9-4762-bd6b-e45636b7e1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:33.627453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453bdfa-315b-4f88-b82d-b5bf18061dd3 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:33.644810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:33.663517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 205/414] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:33.665441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab77f73-63a9-4762-bd6b-e45636b7e1c2 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:33.696925 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05535f-5a25-493a-8121-da1f5a476927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:33.698135 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05535f-5a25-493a-8121-da1f5a476927 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:33.698752 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05535f-5a25-493a-8121-da1f5a476927 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:33.699034 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05535f-5a25-493a-8121-da1f5a476927 None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94/groups/93dde9ef534d4661a2359bb5d1ea03f5/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:33.701403 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05535f-5a25-493a-8121-da1f5a476927 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94, group_id=93dde9ef534d4661a2359bb5d1ea03f5, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:33.703089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05535f-5a25-493a-8121-da1f5a476927 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:33.729472 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 210/415] 10.222.0.76 () {58 vars in 1442 bytes} [Thu May 26 17:50:33 2022] PUT /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94/groups/93dde9ef534d4661a2359bb5d1ea03f5/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:50:33.729472 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a948c-9df1-4bc7-a2a4-ce08b0a5dfe6 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:33.729472 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a948c-9df1-4bc7-a2a4-ce08b0a5dfe6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:33.729472 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a948c-9df1-4bc7-a2a4-ce08b0a5dfe6 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:33.729472 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 211/416] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:33.729472 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a948c-9df1-4bc7-a2a4-ce08b0a5dfe6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:33.811998 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:33.847446 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab77f73-63a9-4762-bd6b-e45636b7e1c2 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:33.914120 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a948c-9df1-4bc7-a2a4-ce08b0a5dfe6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:33.934521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950836a-02da-42cb-aa58-658cfe0e0ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:33.935677 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950836a-02da-42cb-aa58-658cfe0e0ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:33.935995 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950836a-02da-42cb-aa58-658cfe0e0ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:33.936287 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950836a-02da-42cb-aa58-658cfe0e0ee9 None admin] PATH_INFO: `/v3/projects/5a106ee939504b208f8d5ca1deb61f2d` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:33.938530 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6950836a-02da-42cb-aa58-658cfe0e0ee9 None admin] RBAC: Authorizing `identity:get_project(project_id=5a106ee939504b208f8d5ca1deb61f2d)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:33.939206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6950836a-02da-42cb-aa58-658cfe0e0ee9 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:33.941822 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 206/417] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:33 2022] GET /identity/v3/projects/5a106ee939504b208f8d5ca1deb61f2d => generated 344 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:33.945405 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e24994-f926-4682-a80a-c32fd55ca8ea None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:33.945405 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e24994-f926-4682-a80a-c32fd55ca8ea None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:33.945405 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e24994-f926-4682-a80a-c32fd55ca8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:33.977966 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-c3e24994-f926-4682-a80a-c32fd55ca8ea None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:33.977966 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-c3e24994-f926-4682-a80a-c32fd55ca8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:33.989293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5c022-af88-42e9-bad3-8505a7022ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:33.991105 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5c022-af88-42e9-bad3-8505a7022ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:33.991554 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5c022-af88-42e9-bad3-8505a7022ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:33.992355 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5c022-af88-42e9-bad3-8505a7022ed1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:34.013897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5c022-af88-42e9-bad3-8505a7022ed1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:34.029174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-dbe5c022-af88-42e9-bad3-8505a7022ed1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:34.029174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:34.029174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:34.029174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:34.029174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:34.029174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:34.029174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:34.029174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:34.029174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:34.029174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:34.029174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:34.029174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:34.029174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:34.029174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:34.029174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:34.029174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:34.030775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:34.030775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:34.030775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:34.030775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:34.030775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:34.030775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:34.030775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:34.030775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:34.030775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:34.030775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:34.030775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:34.030775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:34.030775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:34.030775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:34.030775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:34.030775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:34.032089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:34.032089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:34.032089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:34.032089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:34.032089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:34.032089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:34.032089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:34.032089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:34.032089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:34.032089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:34.032089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:34.032089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:34.033477 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 212/418] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:34.035010 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5c022-af88-42e9-bad3-8505a7022ed1 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:34.079535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:34.087362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 207/419] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:34.089339 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e24994-f926-4682-a80a-c32fd55ca8ea None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:34.187184 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:34.194458 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5c022-af88-42e9-bad3-8505a7022ed1 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:34.213069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e24994-f926-4682-a80a-c32fd55ca8ea None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:34.235302 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362f750-8d1f-4947-b2c4-b18a596c2d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:34.236101 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362f750-8d1f-4947-b2c4-b18a596c2d64 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:34.236384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362f750-8d1f-4947-b2c4-b18a596c2d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:34.236658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362f750-8d1f-4947-b2c4-b18a596c2d64 None admin] PATH_INFO: `/v3/projects/5a106ee939504b208f8d5ca1deb61f2d/groups/0edbee58926141c2b39b948a6cfe3984/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:34.239364 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362f750-8d1f-4947-b2c4-b18a596c2d64 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a106ee939504b208f8d5ca1deb61f2d, group_id=0edbee58926141c2b39b948a6cfe3984, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:34.240980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362f750-8d1f-4947-b2c4-b18a596c2d64 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:34.260378 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba259c-d00c-4f23-9514-692b85c627f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:34.261584 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba259c-d00c-4f23-9514-692b85c627f9 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:34.262206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba259c-d00c-4f23-9514-692b85c627f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:34.262774 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba259c-d00c-4f23-9514-692b85c627f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:34.263937 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba259c-d00c-4f23-9514-692b85c627f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:34.264616 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba259c-d00c-4f23-9514-692b85c627f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:34.266424 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 213/420] 10.222.0.76 () {58 vars in 1442 bytes} [Thu May 26 17:50:34 2022] PUT /identity/v3/projects/5a106ee939504b208f8d5ca1deb61f2d/groups/0edbee58926141c2b39b948a6cfe3984/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:50:34.266963 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba259c-d00c-4f23-9514-692b85c627f9 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:34.267543 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362f750-8d1f-4947-b2c4-b18a596c2d64 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:34.279358 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 208/421] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:50:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:34.287670 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba259c-d00c-4f23-9514-692b85c627f9 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:34.480012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:34.501885 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362f750-8d1f-4947-b2c4-b18a596c2d64 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:34.566691 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baff142-eed4-4172-8fd5-d32ee9017924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:34.568642 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baff142-eed4-4172-8fd5-d32ee9017924 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:34.569157 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baff142-eed4-4172-8fd5-d32ee9017924 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:34.569455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baff142-eed4-4172-8fd5-d32ee9017924 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:34.585487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baff142-eed4-4172-8fd5-d32ee9017924 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:34.596757 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba259c-d00c-4f23-9514-692b85c627f9 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:34.597457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-5baff142-eed4-4172-8fd5-d32ee9017924 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:50:34.597457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:34.597457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:34.597457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:34.597457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:34.597457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:34.597457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:34.597457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:34.597457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:34.597457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:34.597457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:34.597457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:34.597457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:34.597457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:34.597457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:34.597457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:34.598448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:34.598448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:34.598448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:34.598448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:34.598448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:34.598448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:34.598448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:34.598448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:34.598448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:34.598448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:34.598448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:34.598448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:34.598448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:34.598448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:34.598448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:34.598448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:34.599847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:34.599847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:34.599847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:34.599847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:34.599847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:34.599847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:34.599847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:34.599847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:34.599847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:34.599847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:34.599847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:50:34.599847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:34.599847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 214/422] 10.222.0.76 () {56 vars in 1114 bytes} [Thu May 26 17:50:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:34.607893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baff142-eed4-4172-8fd5-d32ee9017924 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:34.659082 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a091e76-4ccb-4d9a-88f9-a881bf9602a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:34.660561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a091e76-4ccb-4d9a-88f9-a881bf9602a2 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:34.661234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a091e76-4ccb-4d9a-88f9-a881bf9602a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:34.662106 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a091e76-4ccb-4d9a-88f9-a881bf9602a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:34.679577 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:34.750053 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a091e76-4ccb-4d9a-88f9-a881bf9602a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:34.751905 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a091e76-4ccb-4d9a-88f9-a881bf9602a2 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:34.763463 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baff142-eed4-4172-8fd5-d32ee9017924 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:34.817142 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f8ac0d-d231-479f-b7ef-9069567f86e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:34.818099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8ac0d-d231-479f-b7ef-9069567f86e6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:34.818434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8ac0d-d231-479f-b7ef-9069567f86e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:34.818781 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8ac0d-d231-479f-b7ef-9069567f86e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:34.819605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f8ac0d-d231-479f-b7ef-9069567f86e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:34.820071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f8ac0d-d231-479f-b7ef-9069567f86e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:34.821945 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f8ac0d-d231-479f-b7ef-9069567f86e6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:34.832858 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 215/423] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:50:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:34.838839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f8ac0d-d231-479f-b7ef-9069567f86e6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:34.867495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:34.879707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 209/424] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:50:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 17:50:34.891424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a091e76-4ccb-4d9a-88f9-a881bf9602a2 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:34.971269 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:34.981020 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f8ac0d-d231-479f-b7ef-9069567f86e6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:35.024260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1473fac-190c-4055-abbc-e81e6f09376d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:35.025481 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1473fac-190c-4055-abbc-e81e6f09376d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:35.028809 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1473fac-190c-4055-abbc-e81e6f09376d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:35.033158 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1473fac-190c-4055-abbc-e81e6f09376d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:35.108428 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a091e76-4ccb-4d9a-88f9-a881bf9602a2 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:35.151915 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1473fac-190c-4055-abbc-e81e6f09376d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:35.154711 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1473fac-190c-4055-abbc-e81e6f09376d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:35.183998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:35.195737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fbc84-9bd1-4907-8858-e615c121e7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:35.196655 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fbc84-9bd1-4907-8858-e615c121e7ee None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:35.197112 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fbc84-9bd1-4907-8858-e615c121e7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:35.197522 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fbc84-9bd1-4907-8858-e615c121e7ee None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:35.203406 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fbc84-9bd1-4907-8858-e615c121e7ee None admin] RBAC: Authorizing `identity:get_project(project_id=89f44d18dc9b464faeb1bf0bd03569af)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:35.205586 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fbc84-9bd1-4907-8858-e615c121e7ee None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:35.208812 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 210/425] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:34 2022] GET /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af => generated 298 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:35.216674 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fbc84-9bd1-4907-8858-e615c121e7ee None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:35.359302 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 216/426] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:50:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 17:50:35.366460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1473fac-190c-4055-abbc-e81e6f09376d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:35.423371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fbc84-9bd1-4907-8858-e615c121e7ee None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:35.447176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:35.484984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cded2d0-17d7-4490-978b-6d19258d27ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:35.486056 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cded2d0-17d7-4490-978b-6d19258d27ec None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:35.486468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cded2d0-17d7-4490-978b-6d19258d27ec None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:35.486860 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cded2d0-17d7-4490-978b-6d19258d27ec None admin] PATH_INFO: `/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:35.489906 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cded2d0-17d7-4490-978b-6d19258d27ec None admin] RBAC: Authorizing `identity:get_group(group_id=93dde9ef534d4661a2359bb5d1ea03f5)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:35.491650 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cded2d0-17d7-4490-978b-6d19258d27ec None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:35.495748 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 211/427] 10.222.0.76 () {56 vars in 1180 bytes} [Thu May 26 17:50:35 2022] GET /identity/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5 => generated 226 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:35.501070 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cded2d0-17d7-4490-978b-6d19258d27ec None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:35.597940 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1473fac-190c-4055-abbc-e81e6f09376d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:35.633826 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:35.673586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b2dc3-92d0-4d74-b621-dffcde6f76a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:35.674474 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b2dc3-92d0-4d74-b621-dffcde6f76a7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:35.674763 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b2dc3-92d0-4d74-b621-dffcde6f76a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:35.675621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b2dc3-92d0-4d74-b621-dffcde6f76a7 None admin] PATH_INFO: `/v3/projects/89f44d18dc9b464faeb1bf0bd03569af` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:35.678225 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b2dc3-92d0-4d74-b621-dffcde6f76a7 None admin] RBAC: Authorizing `identity:get_project(project_id=89f44d18dc9b464faeb1bf0bd03569af)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:35.679697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b2dc3-92d0-4d74-b621-dffcde6f76a7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:35.684995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 217/428] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:35 2022] GET /identity/v3/projects/89f44d18dc9b464faeb1bf0bd03569af => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:35.692310 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b2dc3-92d0-4d74-b621-dffcde6f76a7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:35.709077 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cded2d0-17d7-4490-978b-6d19258d27ec None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:35.785028 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e3309e-ac5c-4ba5-a46c-c37e9fca3c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:35.785901 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3309e-ac5c-4ba5-a46c-c37e9fca3c30 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:35.786210 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3309e-ac5c-4ba5-a46c-c37e9fca3c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:35.787156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3309e-ac5c-4ba5-a46c-c37e9fca3c30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:35.787909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e3309e-ac5c-4ba5-a46c-c37e9fca3c30 None admin] RBAC: Adding query filter params (group.id=93dde9ef534d4661a2359bb5d1ea03f5, role.id=eb1858a5621543028836223abcb3c0a1, scope.project.id=89f44d18dc9b464faeb1bf0bd03569af) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:35.788061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e3309e-ac5c-4ba5-a46c-c37e9fca3c30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:35.789544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e3309e-ac5c-4ba5-a46c-c37e9fca3c30 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:35.798998 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 212/429] 10.222.0.76 () {56 vars in 1376 bytes} [Thu May 26 17:50:35 2022] GET /identity/v3/role_assignments?group.id=93dde9ef534d4661a2359bb5d1ea03f5&scope.project.id=89f44d18dc9b464faeb1bf0bd03569af&role.id=eb1858a5621543028836223abcb3c0a1 => generated 611 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:35.801019 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7121f13-15bb-40e5-a2de-7ae0b7962029 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:35.801341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7121f13-15bb-40e5-a2de-7ae0b7962029 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:35.802153 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7121f13-15bb-40e5-a2de-7ae0b7962029 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:35.808797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 213/430] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:35.811197 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d450ec-62aa-41c3-a754-d39665a32b26 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:35.811486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d450ec-62aa-41c3-a754-d39665a32b26 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:35.811772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d450ec-62aa-41c3-a754-d39665a32b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:35.851974 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-07d450ec-62aa-41c3-a754-d39665a32b26 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:35.853694 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-07d450ec-62aa-41c3-a754-d39665a32b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:35.896677 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b2dc3-92d0-4d74-b621-dffcde6f76a7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:35.939002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99ddd6-978e-4b20-a344-61aeffd31099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:35.939938 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99ddd6-978e-4b20-a344-61aeffd31099 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:35.940221 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99ddd6-978e-4b20-a344-61aeffd31099 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:35.940546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99ddd6-978e-4b20-a344-61aeffd31099 None admin] PATH_INFO: `/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:35.941910 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e99ddd6-978e-4b20-a344-61aeffd31099 None admin] RBAC: Authorizing `identity:get_group(group_id=93dde9ef534d4661a2359bb5d1ea03f5)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:35.950860 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:35.955529 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e99ddd6-978e-4b20-a344-61aeffd31099 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:35.958009 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 218/431] 10.222.0.76 () {56 vars in 1180 bytes} [Thu May 26 17:50:35 2022] GET /identity/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5 => generated 226 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:35.965755 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99ddd6-978e-4b20-a344-61aeffd31099 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:36.020916 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 214/432] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:36.050804 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d450ec-62aa-41c3-a754-d39665a32b26 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:36.068397 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:36.134762 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99ddd6-978e-4b20-a344-61aeffd31099 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:36.173135 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d450ec-62aa-41c3-a754-d39665a32b26 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:36.181403 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7a5c2-78a0-4dc0-8fdc-529cb687d2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:36.182075 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7a5c2-78a0-4dc0-8fdc-529cb687d2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:36.182392 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7a5c2-78a0-4dc0-8fdc-529cb687d2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:36.182673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7a5c2-78a0-4dc0-8fdc-529cb687d2f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:36.183626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa7a5c2-78a0-4dc0-8fdc-529cb687d2f2 None admin] RBAC: Adding query filter params (group.id=93dde9ef534d4661a2359bb5d1ea03f5, role.id=e731efeb1d3c407fb58cf64a02d95796, scope.project.id=89f44d18dc9b464faeb1bf0bd03569af) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:36.183949 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa7a5c2-78a0-4dc0-8fdc-529cb687d2f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:36.185456 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa7a5c2-78a0-4dc0-8fdc-529cb687d2f2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:36.194152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 219/433] 10.222.0.76 () {56 vars in 1376 bytes} [Thu May 26 17:50:35 2022] GET /identity/v3/role_assignments?group.id=93dde9ef534d4661a2359bb5d1ea03f5&scope.project.id=89f44d18dc9b464faeb1bf0bd03569af&role.id=e731efeb1d3c407fb58cf64a02d95796 => generated 611 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:36.216316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054085ba-c974-40fe-9d50-aa974b6e18ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:36.217244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054085ba-c974-40fe-9d50-aa974b6e18ca None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:36.217553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054085ba-c974-40fe-9d50-aa974b6e18ca None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:36.217949 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054085ba-c974-40fe-9d50-aa974b6e18ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:36.232934 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054085ba-c974-40fe-9d50-aa974b6e18ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:36.241472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-054085ba-c974-40fe-9d50-aa974b6e18ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:50:36.241472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:36.241472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:36.241472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:36.241472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:36.241472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:36.241472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:36.241472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:36.241472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:36.241472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:36.241472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:36.241472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:36.241472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:36.241472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:36.241472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:36.241472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:36.242490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:36.242490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:36.242490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:36.242490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:36.242490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:36.242490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:36.242490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:36.242490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:36.242490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:36.242490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:36.242490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:36.242490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:36.242490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:36.242490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:36.242490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:36.242490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:36.243291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:36.243291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:36.243291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:36.243291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:36.243291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:36.243291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:36.243291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:36.243291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:36.243291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:36.243291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:36.243291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:50:36.243291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:36.243291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 215/434] 10.222.0.76 () {56 vars in 1114 bytes} [Thu May 26 17:50:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:50:36.248668 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7a5c2-78a0-4dc0-8fdc-529cb687d2f2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:36.387550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7a5c2-78a0-4dc0-8fdc-529cb687d2f2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:36.405680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:36.434903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6031118-d201-489b-b8d1-bbac4b3de21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:36.435649 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6031118-d201-489b-b8d1-bbac4b3de21e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:36.435947 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6031118-d201-489b-b8d1-bbac4b3de21e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:36.436229 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6031118-d201-489b-b8d1-bbac4b3de21e None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:36.437012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6031118-d201-489b-b8d1-bbac4b3de21e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:36.437366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6031118-d201-489b-b8d1-bbac4b3de21e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:36.438946 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6031118-d201-489b-b8d1-bbac4b3de21e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:36.451544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 220/435] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:50:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:36.458354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054085ba-c974-40fe-9d50-aa974b6e18ca None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:36.516740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:36.613099 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054085ba-c974-40fe-9d50-aa974b6e18ca None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:36.663400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878aae4-5530-4f58-b745-c9fd7d00793c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:36.664077 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878aae4-5530-4f58-b745-c9fd7d00793c None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:36.664370 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878aae4-5530-4f58-b745-c9fd7d00793c None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:36.664654 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878aae4-5530-4f58-b745-c9fd7d00793c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:36.681417 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69341e9-cc3a-40dc-a192-199d03f972b1 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:36.681749 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69341e9-cc3a-40dc-a192-199d03f972b1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:36.681988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69341e9-cc3a-40dc-a192-199d03f972b1 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:36.684698 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 221/436] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:36.691104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7494022-4ef3-4b26-99a5-0d268110971f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:36.691408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7494022-4ef3-4b26-99a5-0d268110971f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:36.691605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7494022-4ef3-4b26-99a5-0d268110971f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:36.712899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b7494022-4ef3-4b26-99a5-0d268110971f None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:36.716153 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b7494022-4ef3-4b26-99a5-0d268110971f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:36.757536 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878aae4-5530-4f58-b745-c9fd7d00793c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:36.759213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878aae4-5530-4f58-b745-c9fd7d00793c None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:36.813012 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:36.833775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 222/437] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:50:36.856532 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7494022-4ef3-4b26-99a5-0d268110971f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:36.923998 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 216/438] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:50:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:50:36.930527 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878aae4-5530-4f58-b745-c9fd7d00793c None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:37.011436 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7494022-4ef3-4b26-99a5-0d268110971f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:37.022809 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:37.054226 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878aae4-5530-4f58-b745-c9fd7d00793c None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:37.057078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd6584b-a555-4c12-b3d5-1d090f322c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:37.057872 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6584b-a555-4c12-b3d5-1d090f322c58 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:37.058159 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6584b-a555-4c12-b3d5-1d090f322c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:37.058468 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6584b-a555-4c12-b3d5-1d090f322c58 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:37.070390 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd6584b-a555-4c12-b3d5-1d090f322c58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:37.078070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-8bd6584b-a555-4c12-b3d5-1d090f322c58 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:37.078070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:37.078070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:37.078070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:37.078070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:37.078070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:37.078070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:37.078070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:37.078070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:37.078070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:37.078070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:37.078070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:37.078070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:37.078070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:37.078070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:37.078070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:37.079429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:37.079429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:37.079429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:37.079429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:37.079429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:37.079429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:37.079429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:37.079429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:37.079429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:37.079429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:37.079429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:37.079429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:37.079429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:37.079429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:37.079429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:37.079429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:37.080841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:37.080841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:37.080841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:37.080841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:37.080841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:37.080841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:37.080841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:37.080841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:37.080841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:37.080841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:37.080841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 17:50:37.080841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:37.085256 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 223/439] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:50:37.086311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd6584b-a555-4c12-b3d5-1d090f322c58 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:37.103747 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:37.112945 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7987c9-e131-41b4-bfb8-59eb7af28976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:37.114592 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7987c9-e131-41b4-bfb8-59eb7af28976 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:37.115121 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7987c9-e131-41b4-bfb8-59eb7af28976 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:37.115631 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7987c9-e131-41b4-bfb8-59eb7af28976 None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:37.117415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7987c9-e131-41b4-bfb8-59eb7af28976 None admin] RBAC: Authorizing `identity:get_project(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:37.119348 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7987c9-e131-41b4-bfb8-59eb7af28976 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:37.121903 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 217/440] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:36 2022] GET /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:37.127023 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7987c9-e131-41b4-bfb8-59eb7af28976 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:37.238574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd6584b-a555-4c12-b3d5-1d090f322c58 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:37.273937 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7987c9-e131-41b4-bfb8-59eb7af28976 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:37.283222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61dbf4-c492-47bc-9683-9a5f31434c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:37.283907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61dbf4-c492-47bc-9683-9a5f31434c36 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:37.285820 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61dbf4-c492-47bc-9683-9a5f31434c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:37.285820 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61dbf4-c492-47bc-9683-9a5f31434c36 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:37.285820 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61dbf4-c492-47bc-9683-9a5f31434c36 None admin] RBAC: Adding query filter params (name=member) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:37.286501 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61dbf4-c492-47bc-9683-9a5f31434c36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:37.287793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61dbf4-c492-47bc-9683-9a5f31434c36 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:37.310598 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 224/441] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:50:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:37.310598 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fe8de-4eb8-4aec-b520-cad5daac6d15 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:37.310598 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fe8de-4eb8-4aec-b520-cad5daac6d15 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:37.310598 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fe8de-4eb8-4aec-b520-cad5daac6d15 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:37.311361 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 225/442] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:37.312346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27fe8de-4eb8-4aec-b520-cad5daac6d15 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:37.332439 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da752fc-19c9-4952-8b97-2eab63116670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:37.333322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da752fc-19c9-4952-8b97-2eab63116670 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:37.333606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da752fc-19c9-4952-8b97-2eab63116670 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:37.333882 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da752fc-19c9-4952-8b97-2eab63116670 None admin] PATH_INFO: `/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:37.335149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da752fc-19c9-4952-8b97-2eab63116670 None admin] RBAC: Authorizing `identity:get_group(group_id=93dde9ef534d4661a2359bb5d1ea03f5)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:37.336879 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da752fc-19c9-4952-8b97-2eab63116670 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:37.339078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 218/443] 10.222.0.76 () {56 vars in 1180 bytes} [Thu May 26 17:50:37 2022] GET /identity/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:37.340324 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778d27c-7d83-4411-8fd0-9ad8c32a87b7 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:37.340617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778d27c-7d83-4411-8fd0-9ad8c32a87b7 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:37.340896 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778d27c-7d83-4411-8fd0-9ad8c32a87b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:37.347666 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:37.360431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-c778d27c-7d83-4411-8fd0-9ad8c32a87b7 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:37.361645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-c778d27c-7d83-4411-8fd0-9ad8c32a87b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:37.435433 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27fe8de-4eb8-4aec-b520-cad5daac6d15 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:37.435783 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:37.444798 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 219/444] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:37.445930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778d27c-7d83-4411-8fd0-9ad8c32a87b7 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:37.485273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82690552-3f11-4ddb-b6e4-bc01ee51a4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:37.485933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82690552-3f11-4ddb-b6e4-bc01ee51a4fc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:37.486277 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82690552-3f11-4ddb-b6e4-bc01ee51a4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:37.486563 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82690552-3f11-4ddb-b6e4-bc01ee51a4fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:37.582665 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82690552-3f11-4ddb-b6e4-bc01ee51a4fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:37.582877 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778d27c-7d83-4411-8fd0-9ad8c32a87b7 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:37.585081 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82690552-3f11-4ddb-b6e4-bc01ee51a4fc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:37.621384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:37.629560 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90678d25-765e-4f6e-90c0-7ebb21b1c3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:37.630295 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90678d25-765e-4f6e-90c0-7ebb21b1c3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:37.630593 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90678d25-765e-4f6e-90c0-7ebb21b1c3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:37.630917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90678d25-765e-4f6e-90c0-7ebb21b1c3b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:37.631856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90678d25-765e-4f6e-90c0-7ebb21b1c3b2 None admin] RBAC: Adding query filter params (group.id=93dde9ef534d4661a2359bb5d1ea03f5, role.id=e731efeb1d3c407fb58cf64a02d95796, scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:37.632177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90678d25-765e-4f6e-90c0-7ebb21b1c3b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:37.633741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90678d25-765e-4f6e-90c0-7ebb21b1c3b2 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:37.640990 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 220/445] 10.222.0.76 () {56 vars in 1376 bytes} [Thu May 26 17:50:37 2022] GET /identity/v3/role_assignments?group.id=93dde9ef534d4661a2359bb5d1ea03f5&scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94&role.id=e731efeb1d3c407fb58cf64a02d95796 => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:37.641902 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90678d25-765e-4f6e-90c0-7ebb21b1c3b2 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:37.712710 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 226/446] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:50:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:50:37.722231 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82690552-3f11-4ddb-b6e4-bc01ee51a4fc None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:37.736011 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:37.808496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90678d25-765e-4f6e-90c0-7ebb21b1c3b2 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:37.861107 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82690552-3f11-4ddb-b6e4-bc01ee51a4fc None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:37.867708 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86178dfe-db41-45f1-805c-22b4c7b60268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:37.868561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86178dfe-db41-45f1-805c-22b4c7b60268 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:37.868845 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86178dfe-db41-45f1-805c-22b4c7b60268 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:37.869122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86178dfe-db41-45f1-805c-22b4c7b60268 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:37.885131 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86178dfe-db41-45f1-805c-22b4c7b60268 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:37.893228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-86178dfe-db41-45f1-805c-22b4c7b60268 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:50:37.893228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:37.893228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:37.893228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:37.893228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:37.893228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:37.893228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:37.893228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:37.893228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:37.893228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:37.893228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:37.893228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:37.893228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:37.893228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:37.893228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:37.893228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:37.895064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:37.895064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:37.895064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:37.895064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:37.895064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:37.895064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:37.895064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:37.895064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:37.895064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:37.895064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:37.895064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:37.895064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:37.895064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:37.895064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:37.895064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:37.895064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:37.896561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:37.896561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:37.896561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:37.896561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:37.896561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:37.896561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:37.896561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:37.896561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:37.896561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:37.896561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:37.896561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:50:37.896561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:37.896561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 221/447] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:50:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:37.900657 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86178dfe-db41-45f1-805c-22b4c7b60268 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:37.901539 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:37.917234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4726107-ea8d-4505-9792-2f39c7ce3fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:37.918582 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4726107-ea8d-4505-9792-2f39c7ce3fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:37.918968 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4726107-ea8d-4505-9792-2f39c7ce3fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:37.919385 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4726107-ea8d-4505-9792-2f39c7ce3fd3 None admin] PATH_INFO: `/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:37.920876 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4726107-ea8d-4505-9792-2f39c7ce3fd3 None admin] RBAC: Authorizing `identity:get_project(project_id=2c3f7d40141f42ecb82e8c4f7e1b8d94)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:37.923160 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4726107-ea8d-4505-9792-2f39c7ce3fd3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:37.926482 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 227/448] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:37 2022] GET /identity/v3/projects/2c3f7d40141f42ecb82e8c4f7e1b8d94 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:37.934130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4726107-ea8d-4505-9792-2f39c7ce3fd3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:38.018408 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86178dfe-db41-45f1-805c-22b4c7b60268 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:38.042971 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:38.057235 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4726107-ea8d-4505-9792-2f39c7ce3fd3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:38.061156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287122e-24eb-484e-a464-5967371ce2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:38.062124 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287122e-24eb-484e-a464-5967371ce2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:38.062473 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287122e-24eb-484e-a464-5967371ce2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:38.062802 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287122e-24eb-484e-a464-5967371ce2f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:38.064092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287122e-24eb-484e-a464-5967371ce2f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:38.064674 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287122e-24eb-484e-a464-5967371ce2f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:38.067579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287122e-24eb-484e-a464-5967371ce2f6 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:38.079360 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 222/449] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:38.085713 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287122e-24eb-484e-a464-5967371ce2f6 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:38.097408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937dcc88-228d-4c99-ac85-fee6e5b70dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:38.098328 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dcc88-228d-4c99-ac85-fee6e5b70dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:38.098653 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dcc88-228d-4c99-ac85-fee6e5b70dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:38.098887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dcc88-228d-4c99-ac85-fee6e5b70dc9 None admin] PATH_INFO: `/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:38.100151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937dcc88-228d-4c99-ac85-fee6e5b70dc9 None admin] RBAC: Authorizing `identity:get_group(group_id=93dde9ef534d4661a2359bb5d1ea03f5)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:38.101619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937dcc88-228d-4c99-ac85-fee6e5b70dc9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:38.103321 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 228/450] 10.222.0.76 () {56 vars in 1180 bytes} [Thu May 26 17:50:37 2022] GET /identity/v3/groups/93dde9ef534d4661a2359bb5d1ea03f5 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:38.109260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937dcc88-228d-4c99-ac85-fee6e5b70dc9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:38.180014 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:38.205546 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287122e-24eb-484e-a464-5967371ce2f6 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:38.226128 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937dcc88-228d-4c99-ac85-fee6e5b70dc9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:38.244090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7e943-144c-495c-96f2-7c7ac375a4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:38.244755 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7e943-144c-495c-96f2-7c7ac375a4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:38.245041 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7e943-144c-495c-96f2-7c7ac375a4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:38.245323 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7e943-144c-495c-96f2-7c7ac375a4d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:38.275026 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0cb7b-f674-4564-b891-9b6cc9c8c06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:38.276208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0cb7b-f674-4564-b891-9b6cc9c8c06a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:38.276823 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0cb7b-f674-4564-b891-9b6cc9c8c06a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:38.280470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0cb7b-f674-4564-b891-9b6cc9c8c06a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:38.281365 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b0cb7b-f674-4564-b891-9b6cc9c8c06a None admin] RBAC: Adding query filter params (group.id=93dde9ef534d4661a2359bb5d1ea03f5, role.id=eb1858a5621543028836223abcb3c0a1, scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:38.281688 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b0cb7b-f674-4564-b891-9b6cc9c8c06a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:38.283201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b0cb7b-f674-4564-b891-9b6cc9c8c06a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:38.301911 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 229/451] 10.222.0.76 () {56 vars in 1376 bytes} [Thu May 26 17:50:38 2022] GET /identity/v3/role_assignments?group.id=93dde9ef534d4661a2359bb5d1ea03f5&scope.project.id=2c3f7d40141f42ecb82e8c4f7e1b8d94&role.id=eb1858a5621543028836223abcb3c0a1 => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:38.302659 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:38.328210 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7e943-144c-495c-96f2-7c7ac375a4d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:38.329853 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7e943-144c-495c-96f2-7c7ac375a4d5 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:38.458017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 223/452] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:50:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 17:50:38.465840 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0cb7b-f674-4564-b891-9b6cc9c8c06a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:38.602746 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0cb7b-f674-4564-b891-9b6cc9c8c06a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:38.635453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:38.645765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2cc92-ab87-45a4-80d8-27c34f64a582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:38.646697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2cc92-ab87-45a4-80d8-27c34f64a582 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:38.646983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2cc92-ab87-45a4-80d8-27c34f64a582 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:38.647261 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2cc92-ab87-45a4-80d8-27c34f64a582 None admin] PATH_INFO: `/v3/projects/5a106ee939504b208f8d5ca1deb61f2d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:38.648550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2cc92-ab87-45a4-80d8-27c34f64a582 None admin] RBAC: Authorizing `identity:get_project(project_id=5a106ee939504b208f8d5ca1deb61f2d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:38.650038 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2cc92-ab87-45a4-80d8-27c34f64a582 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:38.651824 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 230/453] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:50:38 2022] GET /identity/v3/projects/5a106ee939504b208f8d5ca1deb61f2d => generated 344 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:38.657566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7e943-144c-495c-96f2-7c7ac375a4d5 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:38.764908 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:38.787295 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7e943-144c-495c-96f2-7c7ac375a4d5 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:38.830356 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dee920-124d-4bbc-84d0-247fde45b93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:38.831694 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dee920-124d-4bbc-84d0-247fde45b93f None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:38.832043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dee920-124d-4bbc-84d0-247fde45b93f None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:38.832464 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dee920-124d-4bbc-84d0-247fde45b93f None admin] PATH_INFO: `/v3/groups/0edbee58926141c2b39b948a6cfe3984` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:38.833866 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dee920-124d-4bbc-84d0-247fde45b93f None admin] RBAC: Authorizing `identity:get_group(group_id=0edbee58926141c2b39b948a6cfe3984)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:38.835579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dee920-124d-4bbc-84d0-247fde45b93f None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:38.837430 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 224/454] 10.222.0.76 () {56 vars in 1180 bytes} [Thu May 26 17:50:38 2022] GET /identity/v3/groups/0edbee58926141c2b39b948a6cfe3984 => generated 229 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:38.846737 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2cc92-ab87-45a4-80d8-27c34f64a582 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:38.970447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2cc92-ab87-45a4-80d8-27c34f64a582 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:39.007549 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022643c5-4ba9-472b-89d3-27eb888b6cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:39.008209 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022643c5-4ba9-472b-89d3-27eb888b6cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:39.008493 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022643c5-4ba9-472b-89d3-27eb888b6cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:39.008813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022643c5-4ba9-472b-89d3-27eb888b6cb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:39.009789 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022643c5-4ba9-472b-89d3-27eb888b6cb1 None admin] RBAC: Adding query filter params (group.id=0edbee58926141c2b39b948a6cfe3984, role.id=4fa8f06085c741a5b3cf64e63cc0754f, scope.project.id=5a106ee939504b208f8d5ca1deb61f2d) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:39.010103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022643c5-4ba9-472b-89d3-27eb888b6cb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:39.011589 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022643c5-4ba9-472b-89d3-27eb888b6cb1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:39.018654 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 231/455] 10.222.0.76 () {56 vars in 1376 bytes} [Thu May 26 17:50:38 2022] GET /identity/v3/role_assignments?group.id=0edbee58926141c2b39b948a6cfe3984&scope.project.id=5a106ee939504b208f8d5ca1deb61f2d&role.id=4fa8f06085c741a5b3cf64e63cc0754f => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:42.213215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e792d-7274-4d39-a7dc-247aaddbf7d6 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:42.214221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e792d-7274-4d39-a7dc-247aaddbf7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:42.214716 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e792d-7274-4d39-a7dc-247aaddbf7d6 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:42.216621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 225/456] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:42.222422 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36d450-41c4-490f-b325-dbe2e06ac52c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:42.223404 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36d450-41c4-490f-b325-dbe2e06ac52c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:42.223916 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36d450-41c4-490f-b325-dbe2e06ac52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:42.246865 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ae36d450-41c4-490f-b325-dbe2e06ac52c None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:42.248438 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ae36d450-41c4-490f-b325-dbe2e06ac52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:42.329303 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:42.359047 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 232/457] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:42.404368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e792d-7274-4d39-a7dc-247aaddbf7d6 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:42.570907 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e792d-7274-4d39-a7dc-247aaddbf7d6 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:42.618003 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55dfab-1171-4d5a-af08-3cd57df0a3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:42.618841 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55dfab-1171-4d5a-af08-3cd57df0a3ad None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:42.619129 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55dfab-1171-4d5a-af08-3cd57df0a3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:42.619408 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55dfab-1171-4d5a-af08-3cd57df0a3ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:42.633907 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b55dfab-1171-4d5a-af08-3cd57df0a3ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:42.635410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b55dfab-1171-4d5a-af08-3cd57df0a3ad None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:42.647621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a1372-4b11-4399-b398-7e46a9462e6a None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:42.648257 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a1372-4b11-4399-b398-7e46a9462e6a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:42.648810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a1372-4b11-4399-b398-7e46a9462e6a None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:42.653705 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 233/458] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:42.658517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-3b55dfab-1171-4d5a-af08-3cd57df0a3ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:42.658517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:42.658517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:42.658517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:42.658517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:42.658517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:42.658517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:42.658517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:42.658517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:42.658517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:42.658517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:42.658517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:42.658517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:42.658517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:42.658517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:42.660029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:42.660029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:42.660029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:42.660029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:42.660029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:42.660029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:42.660029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:42.660029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:42.660029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:42.660029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:42.660029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:42.660029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:42.660029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:42.660029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:42.660029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:42.660029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:42.660029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:42.661082 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:42.661082 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:42.661082 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:42.661082 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:42.661082 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 226/459] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:42.663221 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fef4d1-473d-48b5-9606-a477173bce6e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:42.663763 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fef4d1-473d-48b5-9606-a477173bce6e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:42.664376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fef4d1-473d-48b5-9606-a477173bce6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:42.674218 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55dfab-1171-4d5a-af08-3cd57df0a3ad None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:42.674218 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:42.702840 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-75fef4d1-473d-48b5-9606-a477173bce6e None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:42.704863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-75fef4d1-473d-48b5-9606-a477173bce6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:42.787939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55dfab-1171-4d5a-af08-3cd57df0a3ad None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:42.814311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 234/460] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:42.816270 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74334311-b716-4884-a559-15a7c7effe57 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:42.816848 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74334311-b716-4884-a559-15a7c7effe57 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:42.817321 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74334311-b716-4884-a559-15a7c7effe57 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:42.819363 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 235/461] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:42.824811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e5f18-3e59-4d89-8a3b-6170833abcb6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:42.825438 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e5f18-3e59-4d89-8a3b-6170833abcb6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:42.825921 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e5f18-3e59-4d89-8a3b-6170833abcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:42.844569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-089e5f18-3e59-4d89-8a3b-6170833abcb6 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:42.846215 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-089e5f18-3e59-4d89-8a3b-6170833abcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:42.860229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68778d84-cea6-466f-9d3d-6bc1b6340c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:42.860938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68778d84-cea6-466f-9d3d-6bc1b6340c61 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:42.861230 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68778d84-cea6-466f-9d3d-6bc1b6340c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:42.861543 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68778d84-cea6-466f-9d3d-6bc1b6340c61 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:42.862420 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68778d84-cea6-466f-9d3d-6bc1b6340c61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:42.862931 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68778d84-cea6-466f-9d3d-6bc1b6340c61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:42.864819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68778d84-cea6-466f-9d3d-6bc1b6340c61 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:42.886489 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 227/462] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:42.887786 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68778d84-cea6-466f-9d3d-6bc1b6340c61 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:42.931215 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 236/463] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:50:42.938218 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e5f18-3e59-4d89-8a3b-6170833abcb6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:42.962760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:43.026366 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68778d84-cea6-466f-9d3d-6bc1b6340c61 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:43.030609 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:43.076184 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b39b0-e380-4601-a158-5707956d59d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:43.077174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b39b0-e380-4601-a158-5707956d59d7 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:43.077494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b39b0-e380-4601-a158-5707956d59d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:43.077802 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b39b0-e380-4601-a158-5707956d59d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:43.090369 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b39b0-e380-4601-a158-5707956d59d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:43.091950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b39b0-e380-4601-a158-5707956d59d7 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:43.104424 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e5f18-3e59-4d89-8a3b-6170833abcb6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:43.112220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-013b39b0-e380-4601-a158-5707956d59d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:43.112220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:43.112220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:43.112220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:43.112220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:43.112220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:43.112220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:43.112220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:43.112220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:43.112220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:43.112220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:43.112220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:43.112220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:43.112220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:43.112220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:43.114318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:43.114318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:43.114318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:43.114318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:43.114318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:43.114318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:43.114318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:43.114318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:43.114318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:43.114318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:43.114318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:43.114318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:43.114318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:43.114318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:43.114318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:43.114318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:43.114318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:43.115177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:43.115177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:43.115177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:43.115177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:43.115177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 228/464] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:43.115177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b39b0-e380-4601-a158-5707956d59d7 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:43.165610 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1f147-ef57-4932-baef-6e6a8a597419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:43.166650 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1f147-ef57-4932-baef-6e6a8a597419 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:43.166942 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1f147-ef57-4932-baef-6e6a8a597419 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:43.167231 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1f147-ef57-4932-baef-6e6a8a597419 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:43.167996 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1f147-ef57-4932-baef-6e6a8a597419 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:43.169807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1f147-ef57-4932-baef-6e6a8a597419 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:43.207229 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 237/465] 10.222.0.76 () {60 vars in 1132 bytes} [Thu May 26 17:50:42 2022] POST /identity/v3/users => generated 248 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:50:43.209412 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1bba0-687d-4264-a213-42d8f37f2b71 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:43.209901 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1bba0-687d-4264-a213-42d8f37f2b71 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:43.210441 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1bba0-687d-4264-a213-42d8f37f2b71 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:43.212649 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 238/466] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:43.213886 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1bba0-687d-4264-a213-42d8f37f2b71 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:43.278557 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b39b0-e380-4601-a158-5707956d59d7 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:43.302831 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:43.344289 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164553b-ad81-48f1-9f54-e90ebd26d3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:43.345475 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164553b-ad81-48f1-9f54-e90ebd26d3ca None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:43.345824 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164553b-ad81-48f1-9f54-e90ebd26d3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:43.346214 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164553b-ad81-48f1-9f54-e90ebd26d3ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:43.370314 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164553b-ad81-48f1-9f54-e90ebd26d3ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:43.373267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164553b-ad81-48f1-9f54-e90ebd26d3ca None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:43.384133 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1bba0-687d-4264-a213-42d8f37f2b71 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:43.393256 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:43.410315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-6164553b-ad81-48f1-9f54-e90ebd26d3ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:43.410315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:43.410315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:43.410315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:43.410315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:43.410315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:43.410315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:43.410315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:43.410315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:43.410315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:43.410315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:43.410315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:43.410315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:43.410315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:43.410315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:43.412174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:43.412174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:43.412174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:43.412174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:43.412174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:43.412174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:43.412174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:43.412174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:43.412174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:43.412174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:43.412174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:43.412174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:43.412174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:43.412174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:43.412174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:43.412174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:43.412174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:43.413588 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:43.413588 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:43.413588 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:43.413588 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:43.420833 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 229/467] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:43.423417 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49cf8e-32bc-456c-be1c-9b56a7470366 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:43.424047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49cf8e-32bc-456c-be1c-9b56a7470366 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:43.424546 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49cf8e-32bc-456c-be1c-9b56a7470366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:43.432108 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e2131-05cb-47fc-bb40-375016c0eafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:43.432801 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e2131-05cb-47fc-bb40-375016c0eafa None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:43.433088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e2131-05cb-47fc-bb40-375016c0eafa None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:43.433366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e2131-05cb-47fc-bb40-375016c0eafa None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:43.434126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e2131-05cb-47fc-bb40-375016c0eafa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:43.434496 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e2131-05cb-47fc-bb40-375016c0eafa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:43.436148 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e2131-05cb-47fc-bb40-375016c0eafa None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:43.453103 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-6b49cf8e-32bc-456c-be1c-9b56a7470366 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:43.454932 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-6b49cf8e-32bc-456c-be1c-9b56a7470366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:43.458931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 239/468] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:43.459749 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e2131-05cb-47fc-bb40-375016c0eafa None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:43.573729 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 230/469] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:43.574583 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49cf8e-32bc-456c-be1c-9b56a7470366 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:43.622123 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e2131-05cb-47fc-bb40-375016c0eafa None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:43.677826 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9da1b0-361f-431f-bb3e-c9299bcff0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:43.679408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9da1b0-361f-431f-bb3e-c9299bcff0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:43.680014 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9da1b0-361f-431f-bb3e-c9299bcff0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:43.680552 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9da1b0-361f-431f-bb3e-c9299bcff0d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:43.681585 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9da1b0-361f-431f-bb3e-c9299bcff0d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:43.682182 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9da1b0-361f-431f-bb3e-c9299bcff0d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:43.685070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9da1b0-361f-431f-bb3e-c9299bcff0d7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:43.709500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 240/470] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:43.710255 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:43.711309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9da1b0-361f-431f-bb3e-c9299bcff0d7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:43.724582 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49cf8e-32bc-456c-be1c-9b56a7470366 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:43.765134 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:43.765969 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a0df8-ed92-4d42-84f9-ad46a532e94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:43.767340 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a0df8-ed92-4d42-84f9-ad46a532e94e None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:43.767981 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a0df8-ed92-4d42-84f9-ad46a532e94e None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:43.768481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a0df8-ed92-4d42-84f9-ad46a532e94e None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:43.770714 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a0df8-ed92-4d42-84f9-ad46a532e94e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:43.772817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a0df8-ed92-4d42-84f9-ad46a532e94e None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:43.803713 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 231/471] 10.222.0.76 () {60 vars in 1132 bytes} [Thu May 26 17:50:43 2022] POST /identity/v3/users => generated 246 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:50:43.805342 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a0df8-ed92-4d42-84f9-ad46a532e94e None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:43.896277 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9da1b0-361f-431f-bb3e-c9299bcff0d7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:43.950955 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d8a3a-1f5f-4c3c-9019-5221d421fb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:43.951940 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d8a3a-1f5f-4c3c-9019-5221d421fb28 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:43.952225 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d8a3a-1f5f-4c3c-9019-5221d421fb28 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:43.952820 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d8a3a-1f5f-4c3c-9019-5221d421fb28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:43.972384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d8a3a-1f5f-4c3c-9019-5221d421fb28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:43.973882 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a0df8-ed92-4d42-84f9-ad46a532e94e None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:43.974070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d8a3a-1f5f-4c3c-9019-5221d421fb28 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:44.000234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-3f5d8a3a-1f5f-4c3c-9019-5221d421fb28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:44.000234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:44.000234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:44.000234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:44.000234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:44.000234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:44.000234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:44.000234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:44.000234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:44.000234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:44.000234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:44.000234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:44.000234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:44.000234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:44.000234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:44.001163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:44.001163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:44.001163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:44.001163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:44.001163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:44.001163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:44.001163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:44.001163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:44.001163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:44.001163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:44.001163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:44.001163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:44.001163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:44.001163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:44.001163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:44.001163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:44.001163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:44.002024 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:44.002024 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:44.002024 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:44.002024 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:44.002565 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 241/472] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:44.004313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92da685-d01c-4a07-b375-5442d3fbfbb6 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:44.004615 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92da685-d01c-4a07-b375-5442d3fbfbb6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:44.004894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92da685-d01c-4a07-b375-5442d3fbfbb6 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:44.012924 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 242/473] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:44.013955 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92da685-d01c-4a07-b375-5442d3fbfbb6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:44.031274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a79266-5535-4399-a815-2caab1056504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:44.031969 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a79266-5535-4399-a815-2caab1056504 None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:44.032254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a79266-5535-4399-a815-2caab1056504 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:44.032529 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a79266-5535-4399-a815-2caab1056504 None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:44.033250 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a79266-5535-4399-a815-2caab1056504 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:44.036657 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a79266-5535-4399-a815-2caab1056504 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:44.064591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 232/474] 10.222.0.76 () {60 vars in 1132 bytes} [Thu May 26 17:50:43 2022] POST /identity/v3/users => generated 248 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:50:44.066775 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f612c1-de00-4634-b46c-1c875100e4a8 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:44.067251 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f612c1-de00-4634-b46c-1c875100e4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:44.067703 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f612c1-de00-4634-b46c-1c875100e4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:44.089061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-25f612c1-de00-4634-b46c-1c875100e4a8 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:44.090691 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-25f612c1-de00-4634-b46c-1c875100e4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:44.093362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:44.171918 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:44.210874 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92da685-d01c-4a07-b375-5442d3fbfbb6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:44.220402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 233/475] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:44.248329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f612c1-de00-4634-b46c-1c875100e4a8 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:44.267517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46352766-503a-4352-a847-007f36817604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:44.268635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46352766-503a-4352-a847-007f36817604 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:44.269362 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46352766-503a-4352-a847-007f36817604 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:44.270362 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46352766-503a-4352-a847-007f36817604 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:44.271336 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46352766-503a-4352-a847-007f36817604 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:44.272573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46352766-503a-4352-a847-007f36817604 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:44.274625 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46352766-503a-4352-a847-007f36817604 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:44.296800 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 243/476] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:44.303647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46352766-503a-4352-a847-007f36817604 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:44.432873 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f612c1-de00-4634-b46c-1c875100e4a8 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:44.475907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46352766-503a-4352-a847-007f36817604 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:44.487181 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3ebe5-4c42-4aa9-a5f9-68e7eac5921a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:44.487606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3ebe5-4c42-4aa9-a5f9-68e7eac5921a None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:44.487959 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3ebe5-4c42-4aa9-a5f9-68e7eac5921a None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:44.488458 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3ebe5-4c42-4aa9-a5f9-68e7eac5921a None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:44.489892 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3ebe5-4c42-4aa9-a5f9-68e7eac5921a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:44.493276 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3ebe5-4c42-4aa9-a5f9-68e7eac5921a None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:44.501693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.sql.core [None req-e7b3ebe5-4c42-4aa9-a5f9-68e7eac5921a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 17:50:44.501693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 26 17:50:44.501693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [parameters: {'id': '076d40d70e874f5fadf434c0c529cbde', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 26 17:50:44.501693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80260) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 17:50:44.525599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-e7b3ebe5-4c42-4aa9-a5f9-68e7eac5921a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 26 17:50:44.525599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:44.525599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 17:50:44.525599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 17:50:44.525599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 17:50:44.525599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 17:50:44.525599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 17:50:44.525599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application result = self._query(query) May 26 17:50:44.525599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 17:50:44.525599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application conn.query(q) May 26 17:50:44.525599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 17:50:44.525599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 17:50:44.525599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 17:50:44.525599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application result.read() May 26 17:50:44.525599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 17:50:44.527115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 17:50:44.527115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 17:50:44.527115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 17:50:44.527115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 17:50:44.527115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 17:50:44.527115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 17:50:44.527115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 17:50:44.527115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 17:50:44.527115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:44.527115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 17:50:44.527115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:44.527115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:44.527115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 17:50:44.527115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 17:50:44.527115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 26 17:50:44.527115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return ref.to_dict() May 26 17:50:44.527115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 17:50:44.527115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application next(self.gen) May 26 17:50:44.528280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 17:50:44.528280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application yield resource May 26 17:50:44.528280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 17:50:44.528280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application next(self.gen) May 26 17:50:44.528280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 17:50:44.528280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self.session.rollback() May 26 17:50:44.528280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 17:50:44.528280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self.force_reraise() May 26 17:50:44.528280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 17:50:44.528280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise self.value May 26 17:50:44.528280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 17:50:44.528280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 17:50:44.528280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 17:50:44.528280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application session.commit() May 26 17:50:44.528280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 17:50:44.528280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 17:50:44.528280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 17:50:44.529368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._prepare_impl() May 26 17:50:44.529368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 17:50:44.529368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self.session.flush() May 26 17:50:44.529368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 17:50:44.529368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._flush(objects) May 26 17:50:44.529368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 17:50:44.529368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 17:50:44.529368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 17:50:44.529368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application compat.raise_( May 26 17:50:44.529368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 17:50:44.529368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception May 26 17:50:44.529368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 17:50:44.529368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application flush_context.execute() May 26 17:50:44.529368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 17:50:44.529368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rec.execute(self) May 26 17:50:44.529368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 17:50:44.529368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 17:50:44.532136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 17:50:44.532136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 17:50:44.532136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 26 17:50:44.532136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application c = connection._execute_20( May 26 17:50:44.532136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 17:50:44.532136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 17:50:44.532136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 17:50:44.532136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 17:50:44.532136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 17:50:44.532136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 17:50:44.532136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 17:50:44.532136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 17:50:44.532136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 17:50:44.532136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 17:50:44.532136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 17:50:44.532136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception May 26 17:50:44.535687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 17:50:44.535687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 17:50:44.535687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 17:50:44.535687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 17:50:44.535687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 17:50:44.535687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application result = self._query(query) May 26 17:50:44.535687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 17:50:44.535687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application conn.query(q) May 26 17:50:44.535687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 17:50:44.535687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 17:50:44.535687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 17:50:44.535687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application result.read() May 26 17:50:44.535687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 17:50:44.535687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 17:50:44.535687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 17:50:44.535687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 17:50:44.538101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 17:50:44.538101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 17:50:44.538101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 17:50:44.538101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 17:50:44.538101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 17:50:44.538101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 26 17:50:44.538101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [parameters: {'id': '076d40d70e874f5fadf434c0c529cbde', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 26 17:50:44.538101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 17:50:44.538101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:44.538101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:50:44.538101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:44.538101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:44.538101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:44.538101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:44.538101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:44.538101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:44.542202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:44.542202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:44.542202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:44.542202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:44.542202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:44.542202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:44.542202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 26 17:50:44.542202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 26 17:50:44.542202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:44.542202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:44.542202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 26 17:50:44.542202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 26 17:50:44.542202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 17:50:44.542202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 17:50:44.542202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 26 17:50:44.542202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:44.546748 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 234/477] 10.222.0.76 () {60 vars in 1131 bytes} [Thu May 26 17:50:44 2022] POST /identity/v3/roles => generated 144 bytes in 280 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 26 17:50:44.546748 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3ebe5-4c42-4aa9-a5f9-68e7eac5921a None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:44.546748 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:44.555347 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0213ab-8889-41e4-9a68-4517f8be61fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:44.558242 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0213ab-8889-41e4-9a68-4517f8be61fb None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:44.559016 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0213ab-8889-41e4-9a68-4517f8be61fb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:44.560732 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0213ab-8889-41e4-9a68-4517f8be61fb None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:44.562475 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0213ab-8889-41e4-9a68-4517f8be61fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:44.565955 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0213ab-8889-41e4-9a68-4517f8be61fb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:44.584042 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:44.606322 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 244/478] 10.222.0.76 () {60 vars in 1132 bytes} [Thu May 26 17:50:44 2022] POST /identity/v3/users => generated 249 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:50:44.699174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3ebe5-4c42-4aa9-a5f9-68e7eac5921a None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:44.745941 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb292f44-0442-456a-abc1-87dfb2842e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:44.747234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb292f44-0442-456a-abc1-87dfb2842e5d None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:44.747707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb292f44-0442-456a-abc1-87dfb2842e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:44.748380 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb292f44-0442-456a-abc1-87dfb2842e5d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:44.766564 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb292f44-0442-456a-abc1-87dfb2842e5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:44.780735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-fb292f44-0442-456a-abc1-87dfb2842e5d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:50:44.780735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:44.780735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:44.780735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:44.780735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:44.780735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:44.780735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:44.780735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:44.780735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:44.780735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:44.780735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:44.780735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:44.780735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:44.780735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:44.780735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:44.780735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:44.782419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:44.782419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:44.782419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:44.782419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:44.782419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:44.782419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:44.782419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:44.782419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:44.782419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:44.782419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:44.782419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:44.782419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:44.782419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:44.782419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:44.782419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:44.782419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:44.783418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:44.783418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:44.783418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:44.783418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:44.783418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:44.783418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:44.783418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:44.783418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:44.783418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:44.783418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:44.783418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 17:50:44.783418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:44.788878 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 235/479] 10.222.0.76 () {56 vars in 1114 bytes} [Thu May 26 17:50:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:44.799420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0213ab-8889-41e4-9a68-4517f8be61fb None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:44.960380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0213ab-8889-41e4-9a68-4517f8be61fb None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:45.008172 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be50f2-2f45-4873-a9e4-3d318606404c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:45.009307 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be50f2-2f45-4873-a9e4-3d318606404c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:45.009925 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be50f2-2f45-4873-a9e4-3d318606404c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:45.010375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be50f2-2f45-4873-a9e4-3d318606404c None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:45.011389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be50f2-2f45-4873-a9e4-3d318606404c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:45.011846 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be50f2-2f45-4873-a9e4-3d318606404c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:45.013863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be50f2-2f45-4873-a9e4-3d318606404c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:45.025832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 245/480] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:50:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:46.100608 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e4485-ffe0-47d6-a46e-b5cb9016c1f1 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:46.101488 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e4485-ffe0-47d6-a46e-b5cb9016c1f1 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:46.101917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e4485-ffe0-47d6-a46e-b5cb9016c1f1 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:46.103854 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 236/481] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:46.109524 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4f238-22b5-4943-9762-6b0afa76dd3e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:46.110398 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4f238-22b5-4943-9762-6b0afa76dd3e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:46.110830 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4f238-22b5-4943-9762-6b0afa76dd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:46.146236 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-57b4f238-22b5-4943-9762-6b0afa76dd3e None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:46.147487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-57b4f238-22b5-4943-9762-6b0afa76dd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:46.213839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:46.240732 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 246/482] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:46.281612 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e4485-ffe0-47d6-a46e-b5cb9016c1f1 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:46.373252 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:46.434534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e4485-ffe0-47d6-a46e-b5cb9016c1f1 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:46.493017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8caded3-2ef7-4f54-a033-d8b9652b25d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:46.494043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8caded3-2ef7-4f54-a033-d8b9652b25d0 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:46.494524 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8caded3-2ef7-4f54-a033-d8b9652b25d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:46.494891 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8caded3-2ef7-4f54-a033-d8b9652b25d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:46.519552 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8caded3-2ef7-4f54-a033-d8b9652b25d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:46.529984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-a8caded3-2ef7-4f54-a033-d8b9652b25d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:50:46.529984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:46.529984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:46.529984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:46.529984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:46.529984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:46.529984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:46.529984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:46.529984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:46.529984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:46.529984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:46.529984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:46.529984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:46.529984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:46.529984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:46.529984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:46.531291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:46.531291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:46.531291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:46.531291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:46.531291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:46.531291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:46.531291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:46.531291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:46.531291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:46.531291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:46.531291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:46.531291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:46.531291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:46.531291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:46.531291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:46.531291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:46.532450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:46.532450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:46.532450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:46.532450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:46.532450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:46.532450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:46.532450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:46.532450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:46.532450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:46.532450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:46.532450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:50:46.532450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:46.532450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 237/483] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:50:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:46.541376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4f238-22b5-4943-9762-6b0afa76dd3e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:46.736530 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4f238-22b5-4943-9762-6b0afa76dd3e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:46.792664 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3b321-4370-45b3-aa5d-02596afdd4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:46.793526 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3b321-4370-45b3-aa5d-02596afdd4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:46.793930 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3b321-4370-45b3-aa5d-02596afdd4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:46.794338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3b321-4370-45b3-aa5d-02596afdd4d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:46.795271 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc3b321-4370-45b3-aa5d-02596afdd4d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:46.795702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc3b321-4370-45b3-aa5d-02596afdd4d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:46.797304 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc3b321-4370-45b3-aa5d-02596afdd4d8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:46.808132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 247/484] 10.222.0.76 () {56 vars in 1103 bytes} [Thu May 26 17:50:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:46.815550 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8caded3-2ef7-4f54-a033-d8b9652b25d0 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:46.899039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707041e5-48e9-4882-9eb6-97f405bf22be None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:46.899355 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707041e5-48e9-4882-9eb6-97f405bf22be None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:46.899640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707041e5-48e9-4882-9eb6-97f405bf22be None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:46.902957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 248/485] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:50:46.907719 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc284c04-123c-488e-81b3-e7eefce6fa24 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:46.908009 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc284c04-123c-488e-81b3-e7eefce6fa24 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:46.908289 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc284c04-123c-488e-81b3-e7eefce6fa24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:46.930717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-bc284c04-123c-488e-81b3-e7eefce6fa24 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:46.931992 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bc284c04-123c-488e-81b3-e7eefce6fa24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:47.016097 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:47.071538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 249/486] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:50:47.080515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8caded3-2ef7-4f54-a033-d8b9652b25d0 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:47.118561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc284c04-123c-488e-81b3-e7eefce6fa24 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:47.121607 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:47.165619 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178917a-4d21-446a-8939-b5a94f80eed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:47.166510 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178917a-4d21-446a-8939-b5a94f80eed9 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:47.166797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178917a-4d21-446a-8939-b5a94f80eed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:47.167072 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178917a-4d21-446a-8939-b5a94f80eed9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:47.184476 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178917a-4d21-446a-8939-b5a94f80eed9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:47.186082 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178917a-4d21-446a-8939-b5a94f80eed9 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:47.240770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-7178917a-4d21-446a-8939-b5a94f80eed9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:47.240770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:47.240770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:47.240770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:47.240770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:47.240770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:47.240770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:47.240770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:47.240770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:47.240770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:47.240770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:47.240770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:47.240770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:47.240770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:47.240770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:47.243046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:47.243046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:47.243046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:47.243046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:47.243046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:47.243046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:47.243046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:47.243046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:47.243046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:47.243046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:47.243046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:47.243046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:47.243046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:47.243046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:47.243046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:47.243046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:47.243046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:47.245145 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:47.245145 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:47.245145 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:47.245145 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:47.250853 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 238/487] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 436 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:47.257802 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178917a-4d21-446a-8939-b5a94f80eed9 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:47.289201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc284c04-123c-488e-81b3-e7eefce6fa24 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:47.334710 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6204c67-fbdb-4a0a-b756-ed409fcf4c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:47.334710 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6204c67-fbdb-4a0a-b756-ed409fcf4c03 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:47.334710 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6204c67-fbdb-4a0a-b756-ed409fcf4c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:47.334710 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6204c67-fbdb-4a0a-b756-ed409fcf4c03 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:47.343216 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6204c67-fbdb-4a0a-b756-ed409fcf4c03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:47.351797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-e6204c67-fbdb-4a0a-b756-ed409fcf4c03 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:50:47.351797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:47.351797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:47.351797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:47.351797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:47.351797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:47.351797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:47.351797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:47.351797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:47.351797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:47.351797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:47.351797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:47.351797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:47.351797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:47.351797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:47.351797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:47.352765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:47.352765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:47.352765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:47.352765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:47.352765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:47.352765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:47.352765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:47.352765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:47.352765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:47.352765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:47.352765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:47.352765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:47.352765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:47.352765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:47.352765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:47.352765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:47.353568 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:47.353568 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:47.353568 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:47.353568 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:47.353568 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:47.353568 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:47.353568 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:47.353568 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:47.353568 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:47.353568 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:47.353568 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:50:47.353568 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:47.353568 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 250/488] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:50:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:47.360270 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6204c67-fbdb-4a0a-b756-ed409fcf4c03 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:47.440890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:47.472869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178917a-4d21-446a-8939-b5a94f80eed9 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:47.511003 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6204c67-fbdb-4a0a-b756-ed409fcf4c03 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:47.511827 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a1d790-6ddb-4155-a84d-9a9521cdba62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:47.512772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1d790-6ddb-4155-a84d-9a9521cdba62 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:47.513272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1d790-6ddb-4155-a84d-9a9521cdba62 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:47.513769 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1d790-6ddb-4155-a84d-9a9521cdba62 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:47.514814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a1d790-6ddb-4155-a84d-9a9521cdba62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:47.515428 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a1d790-6ddb-4155-a84d-9a9521cdba62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:47.517089 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a1d790-6ddb-4155-a84d-9a9521cdba62 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:47.537072 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 239/489] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:47.539017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617ce39-bad1-441a-9713-c9e8b4c9d6b2 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:47.539608 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617ce39-bad1-441a-9713-c9e8b4c9d6b2 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:47.540107 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617ce39-bad1-441a-9713-c9e8b4c9d6b2 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:47.542101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 240/490] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:47.547366 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617ce39-bad1-441a-9713-c9e8b4c9d6b2 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:47.561772 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75130c25-312f-490e-9ae5-03f9d5df0b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:47.562667 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75130c25-312f-490e-9ae5-03f9d5df0b27 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:47.562958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75130c25-312f-490e-9ae5-03f9d5df0b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:47.563241 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75130c25-312f-490e-9ae5-03f9d5df0b27 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:47.564177 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75130c25-312f-490e-9ae5-03f9d5df0b27 None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:47.564598 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75130c25-312f-490e-9ae5-03f9d5df0b27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:47.566205 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75130c25-312f-490e-9ae5-03f9d5df0b27 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:47.577469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 251/491] 10.222.0.76 () {56 vars in 1103 bytes} [Thu May 26 17:50:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:47.579023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73e626-a994-4f7d-ab97-58fb65e7ffc8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:47.579316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73e626-a994-4f7d-ab97-58fb65e7ffc8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:47.579977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73e626-a994-4f7d-ab97-58fb65e7ffc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:47.592993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:47.604069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8d73e626-a994-4f7d-ab97-58fb65e7ffc8 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:47.605282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8d73e626-a994-4f7d-ab97-58fb65e7ffc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:47.679901 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617ce39-bad1-441a-9713-c9e8b4c9d6b2 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:47.702244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 252/492] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:47.702244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73e626-a994-4f7d-ab97-58fb65e7ffc8 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:47.734550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:47.741820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463ee13-146a-4ddb-951c-1c36abce625c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:47.742990 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463ee13-146a-4ddb-951c-1c36abce625c None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:47.743493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463ee13-146a-4ddb-951c-1c36abce625c None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:47.744013 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463ee13-146a-4ddb-951c-1c36abce625c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:47.754801 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463ee13-146a-4ddb-951c-1c36abce625c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:47.756589 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463ee13-146a-4ddb-951c-1c36abce625c None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:47.773818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-8463ee13-146a-4ddb-951c-1c36abce625c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 17:50:47.773818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:47.773818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:47.773818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:47.773818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:47.773818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:47.773818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:47.773818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:47.773818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:47.773818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:47.773818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:47.773818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:47.773818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:50:47.773818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:50:47.773818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:50:47.773818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:50:47.775017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:47.775017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:47.775017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:50:47.775017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:50:47.775017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:50:47.775017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:50:47.775017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:47.775017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:47.775017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:47.775017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:47.775017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:47.775017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:47.775017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:47.775017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:47.775017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:47.775017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:47.775017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:47.775017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:47.775955 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:47.775955 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:47.775955 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:50:47.775955 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:50:47.775955 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:50:47.775955 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:50:47.775955 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:50:47.775955 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:50:47.775955 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 17:50:47.775955 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:47.783070 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 241/493] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:47.785212 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463ee13-146a-4ddb-951c-1c36abce625c None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:47.826532 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73e626-a994-4f7d-ab97-58fb65e7ffc8 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:47.877567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3334b6-6bbf-4a62-a16f-14038582d4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:47.878948 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3334b6-6bbf-4a62-a16f-14038582d4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:47.879231 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3334b6-6bbf-4a62-a16f-14038582d4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:47.879595 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3334b6-6bbf-4a62-a16f-14038582d4e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:47.894002 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:47.894991 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3334b6-6bbf-4a62-a16f-14038582d4e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:47.896988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3334b6-6bbf-4a62-a16f-14038582d4e9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:47.903523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463ee13-146a-4ddb-951c-1c36abce625c None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:47.921490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-3c3334b6-6bbf-4a62-a16f-14038582d4e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:47.921490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:47.921490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:47.921490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:47.921490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:47.921490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:47.921490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:47.921490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:47.921490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:47.921490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:47.921490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:47.921490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:47.921490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:47.921490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:47.921490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:47.922659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:47.922659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:47.922659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:47.922659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:47.922659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:47.922659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:47.922659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:47.922659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:47.922659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:47.922659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:47.922659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:47.922659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:47.922659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:47.922659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:47.922659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:47.922659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:47.922659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:47.923527 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:47.923527 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:47.923527 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:47.923527 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:47.923527 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 253/494] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:47.924081 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3334b6-6bbf-4a62-a16f-14038582d4e9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:47.945094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18145a9-ffb4-49d2-bcdf-5412ccf7b9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:47.946373 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18145a9-ffb4-49d2-bcdf-5412ccf7b9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:47.946890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18145a9-ffb4-49d2-bcdf-5412ccf7b9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:47.947368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18145a9-ffb4-49d2-bcdf-5412ccf7b9a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:47.959770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18145a9-ffb4-49d2-bcdf-5412ccf7b9a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:47.968726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-f18145a9-ffb4-49d2-bcdf-5412ccf7b9a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:50:47.968726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:47.968726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:47.968726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:47.968726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:47.968726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:47.968726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:47.968726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:47.968726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:47.968726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:47.968726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:47.968726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:47.968726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:47.968726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:47.968726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:47.968726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:47.969658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:47.969658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:47.969658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:47.969658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:47.969658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:47.969658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:47.969658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:47.969658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:47.969658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:47.969658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:47.969658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:47.969658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:47.969658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:47.969658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:47.969658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:47.969658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:47.970523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:47.970523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:47.970523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:47.970523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:47.970523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:47.970523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:47.970523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:47.970523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:47.970523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:47.970523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:47.970523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:50:47.970523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:47.978453 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 242/495] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:50:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:47.979718 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18145a9-ffb4-49d2-bcdf-5412ccf7b9a9 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:48.026400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:48.047451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3334b6-6bbf-4a62-a16f-14038582d4e9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:48.086300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc8a25-d57b-477c-aedf-533238169fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:48.087455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc8a25-d57b-477c-aedf-533238169fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:48.088207 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc8a25-d57b-477c-aedf-533238169fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:48.088939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc8a25-d57b-477c-aedf-533238169fc9 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:48.089972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc8a25-d57b-477c-aedf-533238169fc9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:48.090513 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc8a25-d57b-477c-aedf-533238169fc9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:48.090627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18145a9-ffb4-49d2-bcdf-5412ccf7b9a9 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:48.092435 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc8a25-d57b-477c-aedf-533238169fc9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:48.126259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcee62-71b2-40b9-bf26-5aeb310bfa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:48.126955 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcee62-71b2-40b9-bf26-5aeb310bfa69 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:48.127223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcee62-71b2-40b9-bf26-5aeb310bfa69 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:48.127503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcee62-71b2-40b9-bf26-5aeb310bfa69 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:48.128895 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dcee62-71b2-40b9-bf26-5aeb310bfa69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:48.129208 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dcee62-71b2-40b9-bf26-5aeb310bfa69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:48.130628 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dcee62-71b2-40b9-bf26-5aeb310bfa69 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:48.136707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 254/496] 10.222.0.76 () {56 vars in 1137 bytes} [Thu May 26 17:50:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:48.138912 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc8a25-d57b-477c-aedf-533238169fc9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:48.154974 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 243/497] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:48.154974 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807932ee-e6b4-424c-8979-eed76d670371 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:48.154974 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807932ee-e6b4-424c-8979-eed76d670371 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:48.154974 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807932ee-e6b4-424c-8979-eed76d670371 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:48.155338 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 244/498] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:48.159570 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807932ee-e6b4-424c-8979-eed76d670371 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:48.178033 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:48.248559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc8a25-d57b-477c-aedf-533238169fc9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:48.275510 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807932ee-e6b4-424c-8979-eed76d670371 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:48.283963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7273261-1a43-4f5d-9ee8-0672c0a8d5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:48.285813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7273261-1a43-4f5d-9ee8-0672c0a8d5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:48.286098 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7273261-1a43-4f5d-9ee8-0672c0a8d5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:48.286660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7273261-1a43-4f5d-9ee8-0672c0a8d5f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:48.287429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7273261-1a43-4f5d-9ee8-0672c0a8d5f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:48.287740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7273261-1a43-4f5d-9ee8-0672c0a8d5f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:48.289119 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7273261-1a43-4f5d-9ee8-0672c0a8d5f2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:48.297433 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 255/499] 10.222.0.76 () {56 vars in 1103 bytes} [Thu May 26 17:50:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:48.298338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7273261-1a43-4f5d-9ee8-0672c0a8d5f2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:48.304488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:48.316486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f005a5-9305-46c2-88c6-595ed6d8b2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:48.317141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f005a5-9305-46c2-88c6-595ed6d8b2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:48.317419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f005a5-9305-46c2-88c6-595ed6d8b2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:48.317695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f005a5-9305-46c2-88c6-595ed6d8b2d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:48.385517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f005a5-9305-46c2-88c6-595ed6d8b2d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:48.386925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f005a5-9305-46c2-88c6-595ed6d8b2d7 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:48.403241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:48.420657 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7273261-1a43-4f5d-9ee8-0672c0a8d5f2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:48.457944 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a720a0-496d-4d66-9eb8-51f247eddd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:48.458756 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a720a0-496d-4d66-9eb8-51f247eddd72 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:48.459042 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a720a0-496d-4d66-9eb8-51f247eddd72 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:48.459319 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a720a0-496d-4d66-9eb8-51f247eddd72 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:48.467286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a720a0-496d-4d66-9eb8-51f247eddd72 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:48.468752 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a720a0-496d-4d66-9eb8-51f247eddd72 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:48.484109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-72a720a0-496d-4d66-9eb8-51f247eddd72 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 17:50:48.484109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:48.484109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:48.484109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:48.484109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:48.484109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:48.484109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:48.484109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:48.484109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:48.484109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:48.484109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:48.484109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:48.484109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:50:48.484109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:50:48.484109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:50:48.484109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:50:48.485013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:48.485013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:48.485013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:50:48.485013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:50:48.485013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:50:48.485013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:50:48.485013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:48.485013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:48.485013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:48.485013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:48.485013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:48.485013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:48.485013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:48.485013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:48.485013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:48.485013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:48.485013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:48.485013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:48.485897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:48.485897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:48.485897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:50:48.485897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:50:48.485897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:50:48.485897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:50:48.485897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:50:48.485897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:50:48.485897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 17:50:48.485897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:48.485897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 256/500] 10.222.0.76 () {56 vars in 1093 bytes} [Thu May 26 17:50:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:48.487039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f962983-eaf7-48e3-903e-a0cc947e5a9f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:48.487039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f962983-eaf7-48e3-903e-a0cc947e5a9f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:48.487039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f962983-eaf7-48e3-903e-a0cc947e5a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:48.502349 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 245/501] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:50:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:50:48.505780 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0f962983-eaf7-48e3-903e-a0cc947e5a9f None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:48.507031 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0f962983-eaf7-48e3-903e-a0cc947e5a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:48.507469 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f005a5-9305-46c2-88c6-595ed6d8b2d7 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:48.570966 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:48.582968 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 257/502] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:48.583666 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f962983-eaf7-48e3-903e-a0cc947e5a9f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:48.641348 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f005a5-9305-46c2-88c6-595ed6d8b2d7 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:48.656194 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:48.680463 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa89f05-37d5-4d93-b578-eda219251355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:48.681268 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa89f05-37d5-4d93-b578-eda219251355 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:48.681559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa89f05-37d5-4d93-b578-eda219251355 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:48.681842 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa89f05-37d5-4d93-b578-eda219251355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:48.692294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa89f05-37d5-4d93-b578-eda219251355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:48.693753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa89f05-37d5-4d93-b578-eda219251355 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:48.701627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f962983-eaf7-48e3-903e-a0cc947e5a9f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:48.711402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-0aa89f05-37d5-4d93-b578-eda219251355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:48.711402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:48.711402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:48.711402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:48.711402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:48.711402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:48.711402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:48.711402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:48.711402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:48.711402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:48.711402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:48.711402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:48.711402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:48.711402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:48.711402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:48.712372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:48.712372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:48.712372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:48.712372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:48.712372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:48.712372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:48.712372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:48.712372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:48.712372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:48.712372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:48.712372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:48.712372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:48.712372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:48.712372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:48.712372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:48.712372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:48.712372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:48.713392 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:48.713392 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:48.713392 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:48.713392 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:48.717216 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 246/503] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:48.717910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa89f05-37d5-4d93-b578-eda219251355 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:48.744141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955871b-e5b2-4a31-8398-1a609b01365d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:48.744837 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955871b-e5b2-4a31-8398-1a609b01365d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:48.745124 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955871b-e5b2-4a31-8398-1a609b01365d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:48.745403 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955871b-e5b2-4a31-8398-1a609b01365d None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:48.746359 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955871b-e5b2-4a31-8398-1a609b01365d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:48.746682 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955871b-e5b2-4a31-8398-1a609b01365d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:48.748194 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955871b-e5b2-4a31-8398-1a609b01365d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:48.770314 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 258/504] 10.222.0.76 () {56 vars in 1133 bytes} [Thu May 26 17:50:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:48.775700 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955871b-e5b2-4a31-8398-1a609b01365d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:48.833663 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:48.838738 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa89f05-37d5-4d93-b578-eda219251355 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:48.875210 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb0e16-66d9-42e2-9a77-915c341c6630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:48.876184 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb0e16-66d9-42e2-9a77-915c341c6630 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:48.876564 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb0e16-66d9-42e2-9a77-915c341c6630 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:48.877014 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb0e16-66d9-42e2-9a77-915c341c6630 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:48.888491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb0e16-66d9-42e2-9a77-915c341c6630 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:48.888905 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955871b-e5b2-4a31-8398-1a609b01365d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:48.890234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb0e16-66d9-42e2-9a77-915c341c6630 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:48.911611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-3acb0e16-66d9-42e2-9a77-915c341c6630 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:48.911611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:48.911611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:48.911611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:48.911611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:48.911611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:48.911611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:48.911611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:48.911611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:48.911611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:48.911611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:48.911611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:48.911611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:48.911611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:48.911611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:48.912594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:48.912594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:48.912594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:48.912594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:48.912594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:48.912594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:48.912594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:48.912594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:48.912594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:48.912594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:48.912594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:48.912594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:48.912594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:48.912594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:48.912594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:48.912594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:48.912594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:48.913567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:48.913567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:48.913567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:48.913567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:48.913567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 247/505] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:48.913922 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb0e16-66d9-42e2-9a77-915c341c6630 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:48.930649 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b07cc-11d3-48b9-89c0-c0955d8edc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:48.931941 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b07cc-11d3-48b9-89c0-c0955d8edc87 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:48.932503 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b07cc-11d3-48b9-89c0-c0955d8edc87 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:48.932986 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b07cc-11d3-48b9-89c0-c0955d8edc87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:48.935522 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:48.948567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b07cc-11d3-48b9-89c0-c0955d8edc87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:48.956963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-5e2b07cc-11d3-48b9-89c0-c0955d8edc87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:50:48.956963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:48.956963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:48.956963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:48.956963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:48.956963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:48.956963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:48.956963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:48.956963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:48.956963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:48.956963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:48.956963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:48.956963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:48.956963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:48.956963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:48.956963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:48.958023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:48.958023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:48.958023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:48.958023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:48.958023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:48.958023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:48.958023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:48.958023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:48.958023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:48.958023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:48.958023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:48.958023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:48.958023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:48.958023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:48.958023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:48.958023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:48.964190 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:48.964190 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:48.964190 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:48.964190 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:48.964190 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:48.964190 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:48.964190 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:48.964190 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:48.964190 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:48.964190 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:48.964190 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:50:48.964190 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:48.964190 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 259/506] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:50:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:48.964190 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b07cc-11d3-48b9-89c0-c0955d8edc87 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:49.049286 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb0e16-66d9-42e2-9a77-915c341c6630 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:49.067898 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b07cc-11d3-48b9-89c0-c0955d8edc87 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:49.078765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:49.087002 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dac720-4759-4a4c-a887-0ade987e2a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:49.087665 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dac720-4759-4a4c-a887-0ade987e2a8b None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:49.087947 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dac720-4759-4a4c-a887-0ade987e2a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:49.088223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dac720-4759-4a4c-a887-0ade987e2a8b None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:49.088949 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dac720-4759-4a4c-a887-0ade987e2a8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:49.089258 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dac720-4759-4a4c-a887-0ade987e2a8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:49.090677 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dac720-4759-4a4c-a887-0ade987e2a8b None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:49.102855 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 248/507] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:49.103669 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dac720-4759-4a4c-a887-0ade987e2a8b None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:49.104630 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d166e-3659-4074-b851-3fe9b554be2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:49.105286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d166e-3659-4074-b851-3fe9b554be2d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:49.105573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d166e-3659-4074-b851-3fe9b554be2d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:49.105851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d166e-3659-4074-b851-3fe9b554be2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:49.177176 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d166e-3659-4074-b851-3fe9b554be2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:49.178600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d166e-3659-4074-b851-3fe9b554be2d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:49.186674 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:49.210534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dac720-4759-4a4c-a887-0ade987e2a8b None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:49.249657 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17567a0d-1ac9-4aa2-9dfb-9ca474fbd0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:49.250348 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17567a0d-1ac9-4aa2-9dfb-9ca474fbd0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:49.250633 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17567a0d-1ac9-4aa2-9dfb-9ca474fbd0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:49.250908 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17567a0d-1ac9-4aa2-9dfb-9ca474fbd0a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:49.251627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17567a0d-1ac9-4aa2-9dfb-9ca474fbd0a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:49.251934 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17567a0d-1ac9-4aa2-9dfb-9ca474fbd0a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:49.259117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17567a0d-1ac9-4aa2-9dfb-9ca474fbd0a3 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:49.272468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 249/508] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:49.273290 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17567a0d-1ac9-4aa2-9dfb-9ca474fbd0a3 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:49.284769 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 260/509] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:50:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:50:49.286474 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe998a-a473-49f7-989b-babb82bf4414 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:49.286762 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe998a-a473-49f7-989b-babb82bf4414 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:49.287037 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe998a-a473-49f7-989b-babb82bf4414 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:49.288308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 261/510] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:49.289022 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe998a-a473-49f7-989b-babb82bf4414 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:49.301845 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:49.381139 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17567a0d-1ac9-4aa2-9dfb-9ca474fbd0a3 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:49.395097 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe998a-a473-49f7-989b-babb82bf4414 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:49.418415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc635182-f154-4d98-b96e-2a8e36fbd5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:49.419091 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc635182-f154-4d98-b96e-2a8e36fbd5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:49.419374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc635182-f154-4d98-b96e-2a8e36fbd5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:49.419649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc635182-f154-4d98-b96e-2a8e36fbd5a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:49.420402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc635182-f154-4d98-b96e-2a8e36fbd5a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:49.420711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc635182-f154-4d98-b96e-2a8e36fbd5a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:49.422082 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc635182-f154-4d98-b96e-2a8e36fbd5a3 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:49.427302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:49.430214 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6558d550-43c4-40f0-b258-5414aff7cdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:49.431020 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558d550-43c4-40f0-b258-5414aff7cdb9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:49.431307 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558d550-43c4-40f0-b258-5414aff7cdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:49.431586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558d550-43c4-40f0-b258-5414aff7cdb9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:49.434477 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 250/511] 10.222.0.76 () {56 vars in 1103 bytes} [Thu May 26 17:50:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:49.435214 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc635182-f154-4d98-b96e-2a8e36fbd5a3 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:49.439846 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6558d550-43c4-40f0-b258-5414aff7cdb9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:49.441846 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6558d550-43c4-40f0-b258-5414aff7cdb9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:49.458239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-6558d550-43c4-40f0-b258-5414aff7cdb9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 17:50:49.458239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:49.458239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:49.458239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:49.458239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:49.458239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:49.458239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:49.458239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:49.458239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:49.458239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:49.458239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:49.458239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:49.458239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:50:49.458239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:50:49.458239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:50:49.458239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:50:49.459195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:49.459195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:49.459195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:50:49.459195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:50:49.459195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:50:49.459195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:50:49.459195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:49.459195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:49.459195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:49.459195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:49.459195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:49.459195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:49.459195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:49.459195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:49.459195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:49.459195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:49.459195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:49.459195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:49.460366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:49.460366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:49.460366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:50:49.460366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:50:49.460366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:50:49.460366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:50:49.460366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:50:49.460366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:50:49.460366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 17:50:49.460366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:49.460366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 262/512] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:49.460366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6558d550-43c4-40f0-b258-5414aff7cdb9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:49.540639 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:49.555166 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc635182-f154-4d98-b96e-2a8e36fbd5a3 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:49.583823 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6558d550-43c4-40f0-b258-5414aff7cdb9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:49.595542 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c4d27-6981-4f0a-b7c6-109acf4cf963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:49.596339 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c4d27-6981-4f0a-b7c6-109acf4cf963 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:49.596626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c4d27-6981-4f0a-b7c6-109acf4cf963 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:49.596973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c4d27-6981-4f0a-b7c6-109acf4cf963 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:49.603852 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c4d27-6981-4f0a-b7c6-109acf4cf963 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:49.605331 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c4d27-6981-4f0a-b7c6-109acf4cf963 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:49.619710 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34377c4a-70e3-44e2-9ec6-362aa2215d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:49.620497 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34377c4a-70e3-44e2-9ec6-362aa2215d6d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:49.620772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-e60c4d27-6981-4f0a-b7c6-109acf4cf963 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:50:49.620772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:49.620772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:49.620772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:49.620772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:49.620772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:49.620772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:49.620772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:49.620772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:49.620772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:49.620772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:49.620772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:49.620772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:50:49.620772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:50:49.620772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:50:49.621606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:50:49.621606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:49.621606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:49.621606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:49.621606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:49.621606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:49.621606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:49.621606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:49.621606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:49.621606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:49.621606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:49.621606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:49.621606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:49.621606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:49.621606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:49.621606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:49.621606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:49.622519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:50:49.622519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:50:49.622519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:50:49.622519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:50:49.622519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:50:49.622519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:50:49.622519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:50:49.622519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:49.622937 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34377c4a-70e3-44e2-9ec6-362aa2215d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:49.622937 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34377c4a-70e3-44e2-9ec6-362aa2215d6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:49.623056 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 251/513] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:50:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:49.623564 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8548970-158f-44e4-96c4-9409496981c8 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:49.623809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8548970-158f-44e4-96c4-9409496981c8 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:49.624087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8548970-158f-44e4-96c4-9409496981c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:49.637029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34377c4a-70e3-44e2-9ec6-362aa2215d6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:49.638555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34377c4a-70e3-44e2-9ec6-362aa2215d6d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:49.641647 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-c8548970-158f-44e4-96c4-9409496981c8 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:49.642875 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-c8548970-158f-44e4-96c4-9409496981c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:49.656697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-34377c4a-70e3-44e2-9ec6-362aa2215d6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:49.656697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:49.656697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:49.656697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:49.656697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:49.656697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:49.656697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:49.656697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:49.656697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:49.656697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:49.656697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:49.656697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:49.656697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:49.656697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:49.656697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:49.657740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:49.657740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:49.657740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:49.657740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:49.657740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:49.657740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:49.657740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:49.657740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:49.657740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:49.657740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:49.657740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:49.657740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:49.657740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:49.657740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:49.657740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:49.657740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:49.657740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:49.658846 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:49.658846 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:49.658846 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:49.658846 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:49.658846 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 263/514] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:49.658846 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34377c4a-70e3-44e2-9ec6-362aa2215d6d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:49.703829 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:49.715496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 252/515] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:49.716341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8548970-158f-44e4-96c4-9409496981c8 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:49.766114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34377c4a-70e3-44e2-9ec6-362aa2215d6d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:49.793764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:49.805438 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af0f3a-f86a-454e-b52c-2c5f9dd4f22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:49.806263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af0f3a-f86a-454e-b52c-2c5f9dd4f22f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:49.806550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af0f3a-f86a-454e-b52c-2c5f9dd4f22f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:49.806822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af0f3a-f86a-454e-b52c-2c5f9dd4f22f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:49.818615 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af0f3a-f86a-454e-b52c-2c5f9dd4f22f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:49.820093 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af0f3a-f86a-454e-b52c-2c5f9dd4f22f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:49.824961 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8548970-158f-44e4-96c4-9409496981c8 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:49.837329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-e2af0f3a-f86a-454e-b52c-2c5f9dd4f22f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:49.837329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:49.837329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:49.837329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:49.837329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:49.837329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:49.837329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:49.837329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:49.837329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:49.837329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:49.837329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:49.837329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:49.837329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:49.837329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:49.837329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:49.838388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:49.838388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:49.838388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:49.838388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:49.838388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:49.838388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:49.838388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:49.838388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:49.838388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:49.838388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:49.838388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:49.838388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:49.838388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:49.838388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:49.838388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:49.838388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:49.838388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:49.839410 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:49.839410 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:49.839410 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:49.839410 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:49.839410 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 264/516] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:49.839887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af0f3a-f86a-454e-b52c-2c5f9dd4f22f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:49.861171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3de8cb-febd-48b5-ace4-8c8f3acf2ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:49.861849 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3de8cb-febd-48b5-ace4-8c8f3acf2ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:49.862138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3de8cb-febd-48b5-ace4-8c8f3acf2ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:49.862981 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3de8cb-febd-48b5-ace4-8c8f3acf2ae7 None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:49.863926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3de8cb-febd-48b5-ace4-8c8f3acf2ae7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:49.864245 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3de8cb-febd-48b5-ace4-8c8f3acf2ae7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:49.865709 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3de8cb-febd-48b5-ace4-8c8f3acf2ae7 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:49.903386 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 253/517] 10.222.0.76 () {56 vars in 1137 bytes} [Thu May 26 17:50:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:49.904236 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3de8cb-febd-48b5-ace4-8c8f3acf2ae7 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:49.944961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af0f3a-f86a-454e-b52c-2c5f9dd4f22f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:49.954920 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:49.979742 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac80e92-a8ad-49fd-9ac7-995790af7b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:49.980460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac80e92-a8ad-49fd-9ac7-995790af7b3e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:49.980744 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac80e92-a8ad-49fd-9ac7-995790af7b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:49.981022 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac80e92-a8ad-49fd-9ac7-995790af7b3e None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:49.981815 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac80e92-a8ad-49fd-9ac7-995790af7b3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:49.982127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac80e92-a8ad-49fd-9ac7-995790af7b3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:49.983630 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac80e92-a8ad-49fd-9ac7-995790af7b3e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:50.002762 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 265/518] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:50:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:50.003839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac80e92-a8ad-49fd-9ac7-995790af7b3e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:50.011560 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3de8cb-febd-48b5-ace4-8c8f3acf2ae7 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:50.041378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:50.047783 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8c04d-c9ed-4ca1-8b9d-69e323c3aae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:50.048516 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8c04d-c9ed-4ca1-8b9d-69e323c3aae6 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:50.049472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8c04d-c9ed-4ca1-8b9d-69e323c3aae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:50.050413 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8c04d-c9ed-4ca1-8b9d-69e323c3aae6 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:50.051549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8c04d-c9ed-4ca1-8b9d-69e323c3aae6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:50.051870 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8c04d-c9ed-4ca1-8b9d-69e323c3aae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:50.053264 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8c04d-c9ed-4ca1-8b9d-69e323c3aae6 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:50.065201 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 254/519] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:50.066782 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8c04d-c9ed-4ca1-8b9d-69e323c3aae6 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:50.113391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac80e92-a8ad-49fd-9ac7-995790af7b3e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:50.152823 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b394fca-bd71-448a-95f5-ac9636097fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:50.153630 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b394fca-bd71-448a-95f5-ac9636097fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:50.153916 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b394fca-bd71-448a-95f5-ac9636097fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:50.154214 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b394fca-bd71-448a-95f5-ac9636097fa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:50.165381 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b394fca-bd71-448a-95f5-ac9636097fa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:50.166943 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b394fca-bd71-448a-95f5-ac9636097fa0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:50.174593 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8c04d-c9ed-4ca1-8b9d-69e323c3aae6 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:50.185248 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:50.186544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-5b394fca-bd71-448a-95f5-ac9636097fa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:50.186544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:50.186544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:50.186544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:50.186544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:50.186544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:50.186544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:50.186544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:50.186544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:50.186544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:50.186544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:50.186544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:50.186544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:50.186544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:50.186544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:50.187835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:50.187835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:50.187835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:50.187835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:50.187835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:50.187835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:50.187835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:50.187835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:50.187835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:50.187835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:50.187835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:50.187835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:50.187835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:50.187835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:50.187835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:50.187835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:50.187835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:50.188721 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:50.188721 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:50.188721 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:50.188721 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:50.188721 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 266/520] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:50.188991 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b394fca-bd71-448a-95f5-ac9636097fa0 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:50.217887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44940367-e544-4cb6-a266-513275ebd0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:50.218527 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44940367-e544-4cb6-a266-513275ebd0d4 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:50.218818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44940367-e544-4cb6-a266-513275ebd0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:50.219093 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44940367-e544-4cb6-a266-513275ebd0d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:50.219825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44940367-e544-4cb6-a266-513275ebd0d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:50.220136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44940367-e544-4cb6-a266-513275ebd0d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:50.221534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44940367-e544-4cb6-a266-513275ebd0d4 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:50.234704 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 255/521] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:50.235540 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44940367-e544-4cb6-a266-513275ebd0d4 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:50.293997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:50.311922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b394fca-bd71-448a-95f5-ac9636097fa0 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:50.342214 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44940367-e544-4cb6-a266-513275ebd0d4 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:50.348376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fabed33-ca13-450d-af69-1f9b75d1df91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:50.349031 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabed33-ca13-450d-af69-1f9b75d1df91 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:50.349318 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabed33-ca13-450d-af69-1f9b75d1df91 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:50.349592 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabed33-ca13-450d-af69-1f9b75d1df91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:50.377982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953df96-8e95-49c9-8b00-b4fa7bed59fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:50.378684 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953df96-8e95-49c9-8b00-b4fa7bed59fc None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:50.378920 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953df96-8e95-49c9-8b00-b4fa7bed59fc None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:50.379191 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953df96-8e95-49c9-8b00-b4fa7bed59fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:50.380017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953df96-8e95-49c9-8b00-b4fa7bed59fc None admin] RBAC: Adding query filter params (role.id=e776303775a2423aa0c62c161ecb5a7e, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=52f4c9fc949c4a2c951b755234848cc7) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:50.380333 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953df96-8e95-49c9-8b00-b4fa7bed59fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:50.381774 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953df96-8e95-49c9-8b00-b4fa7bed59fc None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:50.387631 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 256/522] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:50:50 2022] GET /identity/v3/role_assignments?user.id=52f4c9fc949c4a2c951b755234848cc7&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=e776303775a2423aa0c62c161ecb5a7e => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:50.388434 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953df96-8e95-49c9-8b00-b4fa7bed59fc None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:50.431591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:50.440146 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fabed33-ca13-450d-af69-1f9b75d1df91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:50.442130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fabed33-ca13-450d-af69-1f9b75d1df91 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:50.516799 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953df96-8e95-49c9-8b00-b4fa7bed59fc None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:50.561471 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87031047-474b-40fb-9293-fa24131d6543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:50.562317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87031047-474b-40fb-9293-fa24131d6543 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:50.562597 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87031047-474b-40fb-9293-fa24131d6543 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:50.562876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87031047-474b-40fb-9293-fa24131d6543 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:50.563546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:50.571698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87031047-474b-40fb-9293-fa24131d6543 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:50.573186 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87031047-474b-40fb-9293-fa24131d6543 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:50.584338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 267/523] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:50:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:50:50.585521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fabed33-ca13-450d-af69-1f9b75d1df91 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:50.591613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-87031047-474b-40fb-9293-fa24131d6543 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:50:50.591613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:50.591613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:50.591613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:50.591613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:50.591613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:50.591613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:50.591613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:50.591613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:50.591613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:50.591613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:50.591613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:50.591613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:50:50.591613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:50:50.591613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:50:50.592574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:50:50.592574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:50.592574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:50.592574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:50.592574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:50.592574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:50.592574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:50.592574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:50.592574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:50.592574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:50.592574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:50.592574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:50.592574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:50.592574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:50.592574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:50.592574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:50.592574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:50.593422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:50:50.593422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:50:50.593422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:50:50.593422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:50:50.593422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:50:50.593422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:50:50.593422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:50:50.593422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:50.593422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 257/524] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:50:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:50.594363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87031047-474b-40fb-9293-fa24131d6543 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:50.697538 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:50.708104 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87031047-474b-40fb-9293-fa24131d6543 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:50.708417 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fabed33-ca13-450d-af69-1f9b75d1df91 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:50.748132 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a339d0b-b82f-460e-bff9-dfc42aced254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:50.749086 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd1673-025c-4ba7-b4a9-91a0719cac8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:50.749281 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a339d0b-b82f-460e-bff9-dfc42aced254 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:50.749864 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a339d0b-b82f-460e-bff9-dfc42aced254 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:50.750023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd1673-025c-4ba7-b4a9-91a0719cac8b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:50.750151 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a339d0b-b82f-460e-bff9-dfc42aced254 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:50.750509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd1673-025c-4ba7-b4a9-91a0719cac8b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:50.750882 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd1673-025c-4ba7-b4a9-91a0719cac8b None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:50.751682 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd1673-025c-4ba7-b4a9-91a0719cac8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:50.752062 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd1673-025c-4ba7-b4a9-91a0719cac8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:50.754123 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd1673-025c-4ba7-b4a9-91a0719cac8b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:50.759605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a339d0b-b82f-460e-bff9-dfc42aced254 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:50.761263 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a339d0b-b82f-460e-bff9-dfc42aced254 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:50.766573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 268/525] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:50.767742 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd1673-025c-4ba7-b4a9-91a0719cac8b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:50.780062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-4a339d0b-b82f-460e-bff9-dfc42aced254 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 17:50:50.780062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:50.780062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:50.780062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:50.780062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:50.780062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:50.780062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:50.780062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:50.780062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:50.780062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:50.780062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:50.780062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:50.780062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:50:50.780062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:50:50.780062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:50:50.780062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:50:50.781501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:50.781501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:50.781501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:50:50.781501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:50:50.781501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:50:50.781501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:50:50.781501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:50.781501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:50.781501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:50.781501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:50.781501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:50.781501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:50.781501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:50.781501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:50.781501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:50.781501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:50.781501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:50.781501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:50.782680 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:50.782680 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:50.782680 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:50:50.782680 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:50:50.782680 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:50:50.782680 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:50:50.782680 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:50:50.782680 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:50:50.782680 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 17:50:50.782680 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:50.789235 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 258/526] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:50:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:50.789235 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a339d0b-b82f-460e-bff9-dfc42aced254 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:50.830068 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:50.894899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd1673-025c-4ba7-b4a9-91a0719cac8b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:50.910335 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a339d0b-b82f-460e-bff9-dfc42aced254 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:50.935775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d3030a-4cee-40ac-bbd6-d785c65c0b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:50.936729 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3030a-4cee-40ac-bbd6-d785c65c0b2c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:50.937020 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3030a-4cee-40ac-bbd6-d785c65c0b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:50.937299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3030a-4cee-40ac-bbd6-d785c65c0b2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:50.949856 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d3030a-4cee-40ac-bbd6-d785c65c0b2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:50.950477 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4b818-f8d0-4ec9-bdf1-c7517cd03eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:50.951152 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4b818-f8d0-4ec9-bdf1-c7517cd03eeb None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:50.951384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d3030a-4cee-40ac-bbd6-d785c65c0b2c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:50.951504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4b818-f8d0-4ec9-bdf1-c7517cd03eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:50.951702 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4b818-f8d0-4ec9-bdf1-c7517cd03eeb None admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:50.952687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa4b818-f8d0-4ec9-bdf1-c7517cd03eeb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:50.953006 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa4b818-f8d0-4ec9-bdf1-c7517cd03eeb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:50.954471 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa4b818-f8d0-4ec9-bdf1-c7517cd03eeb None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:50.971995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-20d3030a-4cee-40ac-bbd6-d785c65c0b2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:50.971995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:50.971995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:50.971995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:50.971995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:50.971995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:50.971995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:50.971995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:50.971995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:50.971995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:50.971995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:50.971995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:50.971995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:50.971995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:50.971995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:50.972963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:50.972963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:50.972963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:50.972963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:50.972963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:50.972963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:50.972963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:50.972963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:50.972963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:50.972963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:50.972963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:50.972963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:50.972963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:50.972963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:50.972963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:50.972963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:50.972963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:50.973953 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:50.973953 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:50.973953 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:50.973953 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:50.973953 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 269/527] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:50.974963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d3030a-4cee-40ac-bbd6-d785c65c0b2c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:50.977854 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 259/528] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:50:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:50.982233 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4b818-f8d0-4ec9-bdf1-c7517cd03eeb None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:50.997700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:51.098966 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:51.100380 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4b818-f8d0-4ec9-bdf1-c7517cd03eeb None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:51.103464 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d3030a-4cee-40ac-bbd6-d785c65c0b2c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:51.141597 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2bbcd1-80c6-4a28-af18-bd01b2710650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:51.142416 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bbcd1-80c6-4a28-af18-bd01b2710650 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:51.142767 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bbcd1-80c6-4a28-af18-bd01b2710650 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:51.143116 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bbcd1-80c6-4a28-af18-bd01b2710650 None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:51.144196 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2bbcd1-80c6-4a28-af18-bd01b2710650 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:51.144578 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2bbcd1-80c6-4a28-af18-bd01b2710650 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:51.146249 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2bbcd1-80c6-4a28-af18-bd01b2710650 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:51.151461 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169318aa-f67f-4f34-8578-79c85fd81943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:51.152240 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169318aa-f67f-4f34-8578-79c85fd81943 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:51.152593 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169318aa-f67f-4f34-8578-79c85fd81943 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:51.152948 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169318aa-f67f-4f34-8578-79c85fd81943 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:51.153741 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169318aa-f67f-4f34-8578-79c85fd81943 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:51.155317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169318aa-f67f-4f34-8578-79c85fd81943 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:51.165330 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 260/529] 10.222.0.76 () {56 vars in 1139 bytes} [Thu May 26 17:50:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:51.170976 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2bbcd1-80c6-4a28-af18-bd01b2710650 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:51.177731 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 270/530] 10.222.0.76 () {60 vars in 1132 bytes} [Thu May 26 17:50:50 2022] POST /identity/v3/users => generated 247 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:50:51.178419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169318aa-f67f-4f34-8578-79c85fd81943 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:51.291858 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:51.319864 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169318aa-f67f-4f34-8578-79c85fd81943 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:51.341874 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2bbcd1-80c6-4a28-af18-bd01b2710650 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:51.371700 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008715f-dd52-4f89-a9e7-6d10172b232c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:51.372672 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008715f-dd52-4f89-a9e7-6d10172b232c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:51.372967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008715f-dd52-4f89-a9e7-6d10172b232c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:51.373257 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008715f-dd52-4f89-a9e7-6d10172b232c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:51.374972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6008715f-dd52-4f89-a9e7-6d10172b232c None admin] RBAC: Adding query filter params (role.id=e776303775a2423aa0c62c161ecb5a7e, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=1899c317083d40f5b490cd9eb63fe521) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:51.375082 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6008715f-dd52-4f89-a9e7-6d10172b232c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:51.377392 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6008715f-dd52-4f89-a9e7-6d10172b232c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:51.384405 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 271/531] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:50:51 2022] GET /identity/v3/role_assignments?user.id=1899c317083d40f5b490cd9eb63fe521&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=e776303775a2423aa0c62c161ecb5a7e => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:51.387910 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008715f-dd52-4f89-a9e7-6d10172b232c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:51.388688 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494b6c9-9e86-496e-82a0-fa9b6a46707c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:51.389459 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494b6c9-9e86-496e-82a0-fa9b6a46707c None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:51.389819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494b6c9-9e86-496e-82a0-fa9b6a46707c None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:51.390271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494b6c9-9e86-496e-82a0-fa9b6a46707c None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:51.391082 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494b6c9-9e86-496e-82a0-fa9b6a46707c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:51.391487 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494b6c9-9e86-496e-82a0-fa9b6a46707c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:51.393079 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494b6c9-9e86-496e-82a0-fa9b6a46707c None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:51.407773 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:51.410856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 261/532] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:51.416996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494b6c9-9e86-496e-82a0-fa9b6a46707c None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:51.513090 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008715f-dd52-4f89-a9e7-6d10172b232c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:51.554435 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c539cfd-948f-49f4-ac8c-f25f0c496a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:51.555204 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c539cfd-948f-49f4-ac8c-f25f0c496a86 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:51.555539 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c539cfd-948f-49f4-ac8c-f25f0c496a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:51.555818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c539cfd-948f-49f4-ac8c-f25f0c496a86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:51.563133 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494b6c9-9e86-496e-82a0-fa9b6a46707c None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:51.626117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:51.639229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab1201-4f64-40cb-bd73-52d6cdd11763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:51.640658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab1201-4f64-40cb-bd73-52d6cdd11763 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:51.641180 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab1201-4f64-40cb-bd73-52d6cdd11763 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:51.641660 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab1201-4f64-40cb-bd73-52d6cdd11763 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:51.641766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c539cfd-948f-49f4-ac8c-f25f0c496a86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:51.643287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c539cfd-948f-49f4-ac8c-f25f0c496a86 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:51.657578 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab1201-4f64-40cb-bd73-52d6cdd11763 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:51.661539 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab1201-4f64-40cb-bd73-52d6cdd11763 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:51.664630 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:51.694443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-d3ab1201-4f64-40cb-bd73-52d6cdd11763 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:50:51.694443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:51.694443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:51.694443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:51.694443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:51.694443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:51.694443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:51.694443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:51.694443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:51.694443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:51.694443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:51.694443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:51.694443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:50:51.694443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:50:51.694443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:50:51.695363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:50:51.695363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:51.695363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:51.695363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:51.695363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:51.695363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:51.695363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:51.695363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:51.695363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:51.695363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:51.695363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:51.695363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:51.695363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:51.695363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:51.695363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:51.695363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:51.695363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:51.696225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:50:51.696225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:50:51.696225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:50:51.696225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:50:51.696225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:50:51.696225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:50:51.696225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:50:51.696225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:51.696225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 262/533] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:50:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:50:51.701620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab1201-4f64-40cb-bd73-52d6cdd11763 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:51.792806 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 272/534] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:50:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:50:51.796454 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c539cfd-948f-49f4-ac8c-f25f0c496a86 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:51.882531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab1201-4f64-40cb-bd73-52d6cdd11763 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:51.921496 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c539cfd-948f-49f4-ac8c-f25f0c496a86 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:51.948976 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420000a-dc48-49a9-bdef-061cda4047db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:51.948976 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420000a-dc48-49a9-bdef-061cda4047db None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:51.948976 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420000a-dc48-49a9-bdef-061cda4047db None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:51.948976 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420000a-dc48-49a9-bdef-061cda4047db None admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:51.949393 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420000a-dc48-49a9-bdef-061cda4047db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:51.949393 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420000a-dc48-49a9-bdef-061cda4047db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:51.949393 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420000a-dc48-49a9-bdef-061cda4047db None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:51.957834 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:51.966226 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 263/535] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:50:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:50:51.970611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420000a-dc48-49a9-bdef-061cda4047db None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:51.994592 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c9792-0160-496b-b413-d87baeb10ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:51.995706 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c9792-0160-496b-b413-d87baeb10ebb None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:51.996169 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c9792-0160-496b-b413-d87baeb10ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:51.996621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c9792-0160-496b-b413-d87baeb10ebb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:52.021551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c9792-0160-496b-b413-d87baeb10ebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:52.023165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c9792-0160-496b-b413-d87baeb10ebb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:52.047442 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:52.068414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-347c9792-0160-496b-b413-d87baeb10ebb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:52.068414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:52.068414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:52.068414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:52.068414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:52.068414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:52.068414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:52.068414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:52.068414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:52.068414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:52.068414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:52.068414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:52.068414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:52.068414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:52.068414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:52.069426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:52.069426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:52.069426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:52.069426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:52.069426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:52.069426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:52.069426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:52.069426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:52.069426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:52.069426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:52.069426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:52.069426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:52.069426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:52.069426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:52.069426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:52.069426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:52.069426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:52.070316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:52.070316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:52.070316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:52.070316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:52.072920 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 273/536] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:52.078333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c9792-0160-496b-b413-d87baeb10ebb None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:52.164940 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420000a-dc48-49a9-bdef-061cda4047db None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:52.216936 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c9792-0160-496b-b413-d87baeb10ebb None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:52.217771 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1720d29-c6ec-4890-a63a-f583c0a892ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:52.218480 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1720d29-c6ec-4890-a63a-f583c0a892ed None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:52.218767 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1720d29-c6ec-4890-a63a-f583c0a892ed None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:52.219047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1720d29-c6ec-4890-a63a-f583c0a892ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:52.219889 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1720d29-c6ec-4890-a63a-f583c0a892ed None admin] RBAC: Adding query filter params (role.id=e776303775a2423aa0c62c161ecb5a7e, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=76dbf8f3f1b44baca2d2aa90fe606a30) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:52.220205 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1720d29-c6ec-4890-a63a-f583c0a892ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:52.221735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1720d29-c6ec-4890-a63a-f583c0a892ed None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:52.237949 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 264/537] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:50:51 2022] GET /identity/v3/role_assignments?user.id=76dbf8f3f1b44baca2d2aa90fe606a30&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=e776303775a2423aa0c62c161ecb5a7e => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:52.270487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d7b99-620d-4fff-8b7b-668c91f9ed8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:52.271177 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d7b99-620d-4fff-8b7b-668c91f9ed8d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:52.271470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d7b99-620d-4fff-8b7b-668c91f9ed8d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:52.271746 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d7b99-620d-4fff-8b7b-668c91f9ed8d None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:52.272479 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d7b99-620d-4fff-8b7b-668c91f9ed8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:52.272786 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d7b99-620d-4fff-8b7b-668c91f9ed8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:52.274182 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d7b99-620d-4fff-8b7b-668c91f9ed8d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:52.295198 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 274/538] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:52.301836 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1720d29-c6ec-4890-a63a-f583c0a892ed None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:52.439590 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1720d29-c6ec-4890-a63a-f583c0a892ed None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:52.458006 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:52.496050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ee391-2e14-41e3-a906-5d41bfc0bfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:52.496886 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ee391-2e14-41e3-a906-5d41bfc0bfd0 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:52.497167 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ee391-2e14-41e3-a906-5d41bfc0bfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:52.497443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ee391-2e14-41e3-a906-5d41bfc0bfd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:52.506031 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ee391-2e14-41e3-a906-5d41bfc0bfd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:52.507558 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ee391-2e14-41e3-a906-5d41bfc0bfd0 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:52.529770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-9e9ee391-2e14-41e3-a906-5d41bfc0bfd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:50:52.529770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:52.529770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:52.529770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:52.529770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:52.529770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:52.529770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:52.529770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:52.529770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:52.529770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:52.529770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:52.529770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:52.529770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:50:52.529770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:50:52.529770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:50:52.530772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:50:52.530772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:52.530772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:52.530772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:52.530772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:52.530772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:52.530772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:52.530772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:52.530772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:52.530772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:52.530772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:52.530772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:52.530772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:52.530772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:52.530772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:52.530772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:52.530772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:52.531653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:50:52.531653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:50:52.531653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:50:52.531653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:50:52.531653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:50:52.531653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:50:52.531653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:50:52.531653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:52.531653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 265/539] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:50:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:52.537662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d7b99-620d-4fff-8b7b-668c91f9ed8d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:52.635687 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:52.738660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d7b99-620d-4fff-8b7b-668c91f9ed8d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:52.787141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdbd3de-46db-422b-b628-ea4587a6969c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:52.787835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbd3de-46db-422b-b628-ea4587a6969c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:52.788112 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbd3de-46db-422b-b628-ea4587a6969c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:52.788394 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbd3de-46db-422b-b628-ea4587a6969c None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:52.789184 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdbd3de-46db-422b-b628-ea4587a6969c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:52.789504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdbd3de-46db-422b-b628-ea4587a6969c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:52.791064 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdbd3de-46db-422b-b628-ea4587a6969c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:52.810620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 275/540] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:50:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:52.816914 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ee391-2e14-41e3-a906-5d41bfc0bfd0 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:52.979217 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad2dcb-c067-41d6-8b2d-18a3a75568ac None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:52.979580 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad2dcb-c067-41d6-8b2d-18a3a75568ac None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:52.979778 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad2dcb-c067-41d6-8b2d-18a3a75568ac None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:52.981447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 276/541] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:53.016149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ee391-2e14-41e3-a906-5d41bfc0bfd0 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:53.023717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de05c98-697f-4862-9139-0a4584bd48c7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:53.023998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de05c98-697f-4862-9139-0a4584bd48c7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:53.024278 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de05c98-697f-4862-9139-0a4584bd48c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:53.063326 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bce13f-f677-45ac-823d-a0bc8327408b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:53.064002 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bce13f-f677-45ac-823d-a0bc8327408b None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:53.064291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bce13f-f677-45ac-823d-a0bc8327408b None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:53.064572 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bce13f-f677-45ac-823d-a0bc8327408b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:53.066046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bce13f-f677-45ac-823d-a0bc8327408b None admin] RBAC: Adding query filter params (role.id=e776303775a2423aa0c62c161ecb5a7e, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=8df66990a3734209a350c8b6768d259f) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:53.066611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bce13f-f677-45ac-823d-a0bc8327408b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:53.068272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bce13f-f677-45ac-823d-a0bc8327408b None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:53.071232 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-9de05c98-697f-4862-9139-0a4584bd48c7 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:53.072453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9de05c98-697f-4862-9139-0a4584bd48c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:53.074789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 266/542] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:50:52 2022] GET /identity/v3/role_assignments?user.id=8df66990a3734209a350c8b6768d259f&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=e776303775a2423aa0c62c161ecb5a7e => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:53.208958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 277/543] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:53.244573 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bce13f-f677-45ac-823d-a0bc8327408b None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:53.307949 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:53.393276 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bce13f-f677-45ac-823d-a0bc8327408b None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:53.431961 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47935c7e-8029-46cc-815c-61fd0d9c0ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:53.432763 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47935c7e-8029-46cc-815c-61fd0d9c0ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:53.433047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47935c7e-8029-46cc-815c-61fd0d9c0ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:53.433322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47935c7e-8029-46cc-815c-61fd0d9c0ba5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:53.445617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47935c7e-8029-46cc-815c-61fd0d9c0ba5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:53.453610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-47935c7e-8029-46cc-815c-61fd0d9c0ba5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:50:53.453610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:53.453610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:53.453610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:53.453610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:53.453610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:53.453610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:53.453610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:53.453610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:53.453610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:53.453610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:53.453610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:53.453610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:53.453610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:53.453610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:53.453610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:53.454627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:53.454627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:53.454627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:53.454627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:53.454627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:53.454627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:53.454627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:53.454627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:53.454627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:53.454627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:53.454627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:53.454627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:53.454627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:53.454627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:53.454627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:53.454627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:53.455448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:53.455448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:53.455448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:53.455448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:53.455448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:53.455448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:53.455448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:53.455448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:53.455448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:53.455448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:53.455448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:50:53.455448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:53.455448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 267/544] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:50:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:53.462832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de05c98-697f-4862-9139-0a4584bd48c7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:53.491442 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:53.606219 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de05c98-697f-4862-9139-0a4584bd48c7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:53.656062 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fca73-78f7-44c3-97ef-7fb01a608f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:53.656748 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fca73-78f7-44c3-97ef-7fb01a608f1b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:53.657032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fca73-78f7-44c3-97ef-7fb01a608f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:53.657310 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fca73-78f7-44c3-97ef-7fb01a608f1b None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:53.658059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9fca73-78f7-44c3-97ef-7fb01a608f1b None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:53.658390 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9fca73-78f7-44c3-97ef-7fb01a608f1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:53.660341 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9fca73-78f7-44c3-97ef-7fb01a608f1b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:53.673992 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 278/545] 10.222.0.76 () {56 vars in 1103 bytes} [Thu May 26 17:50:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:53.680018 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47935c7e-8029-46cc-815c-61fd0d9c0ba5 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:53.838050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:53.847789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47935c7e-8029-46cc-815c-61fd0d9c0ba5 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:53.897098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086cd52-2e7e-4e5d-9037-0aec4a50e4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:53.897831 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086cd52-2e7e-4e5d-9037-0aec4a50e4bb None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:53.898117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086cd52-2e7e-4e5d-9037-0aec4a50e4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:53.898424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086cd52-2e7e-4e5d-9037-0aec4a50e4bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:53.919284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086cd52-2e7e-4e5d-9037-0aec4a50e4bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:53.920759 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086cd52-2e7e-4e5d-9037-0aec4a50e4bb None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:53.941431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-f086cd52-2e7e-4e5d-9037-0aec4a50e4bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:53.941431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:53.941431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:53.941431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:53.941431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:53.941431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:53.941431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:53.941431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:53.941431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:53.941431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:53.941431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:53.941431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:53.941431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:53.941431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:53.941431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:53.942381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:53.942381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:53.942381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:53.942381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:53.942381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:53.942381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:53.942381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:53.942381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:53.942381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:53.942381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:53.942381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:53.942381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:53.942381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:53.942381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:53.942381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:53.942381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:53.942381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:53.943318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:53.943318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:53.943318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:53.943318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:53.943318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 268/546] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:53.951299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fca73-78f7-44c3-97ef-7fb01a608f1b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:54.099402 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:54.126878 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fca73-78f7-44c3-97ef-7fb01a608f1b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:54.183909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e751a-402a-4a6c-a5fb-78a7981c90f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:54.184508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e751a-402a-4a6c-a5fb-78a7981c90f5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:54.184793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e751a-402a-4a6c-a5fb-78a7981c90f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:54.185072 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e751a-402a-4a6c-a5fb-78a7981c90f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:54.185794 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e751a-402a-4a6c-a5fb-78a7981c90f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:54.186102 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e751a-402a-4a6c-a5fb-78a7981c90f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:54.187520 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e751a-402a-4a6c-a5fb-78a7981c90f5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:54.207278 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 279/547] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:54.212886 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086cd52-2e7e-4e5d-9037-0aec4a50e4bb None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:54.296609 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cd11b-dfca-4a4c-9762-e706180525fb None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:54.296924 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cd11b-dfca-4a4c-9762-e706180525fb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:54.297181 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cd11b-dfca-4a4c-9762-e706180525fb None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:54.298849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 280/548] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:54.310130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e41d7-ff9f-4b50-bf31-38590e6416b3 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:54.310446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e41d7-ff9f-4b50-bf31-38590e6416b3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:54.310731 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e41d7-ff9f-4b50-bf31-38590e6416b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:54.348891 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f54e41d7-ff9f-4b50-bf31-38590e6416b3 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:54.350412 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f54e41d7-ff9f-4b50-bf31-38590e6416b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:54.383123 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086cd52-2e7e-4e5d-9037-0aec4a50e4bb None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:54.427064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e6c6d-0136-4b0c-8855-5844e9756506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:54.428033 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e6c6d-0136-4b0c-8855-5844e9756506 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:54.428390 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e6c6d-0136-4b0c-8855-5844e9756506 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:54.428925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e6c6d-0136-4b0c-8855-5844e9756506 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:54.438305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e6c6d-0136-4b0c-8855-5844e9756506 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:54.439733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e6c6d-0136-4b0c-8855-5844e9756506 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:54.447766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 281/549] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:50:54.454460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdc358-a250-47c1-8972-8bf563e9244f None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:54.454460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdc358-a250-47c1-8972-8bf563e9244f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:54.454460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdc358-a250-47c1-8972-8bf563e9244f None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:54.454460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 282/550] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:54.465670 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d5827-56c8-47bc-914e-8d0d8c17914d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:54.465966 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d5827-56c8-47bc-914e-8d0d8c17914d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:54.466265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d5827-56c8-47bc-914e-8d0d8c17914d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:54.470910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-427e6c6d-0136-4b0c-8855-5844e9756506 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 17:50:54.470910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:54.470910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:54.470910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:54.470910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:54.470910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:54.470910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:54.470910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:54.470910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:54.470910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:54.470910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:54.470910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:54.470910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:50:54.470910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:50:54.470910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:50:54.470910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:50:54.472519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:54.472519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:54.472519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:50:54.472519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:50:54.472519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:50:54.472519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:50:54.472519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:54.472519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:54.472519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:54.472519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:54.472519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:54.472519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:54.472519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:54.472519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:54.472519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:54.472519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:54.472519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:54.472519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:54.473448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:54.473448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:54.473448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:50:54.473448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:50:54.473448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:50:54.473448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:50:54.473448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:50:54.473448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:50:54.473448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 17:50:54.473448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:54.473448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 269/551] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:54.479022 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e6c6d-0136-4b0c-8855-5844e9756506 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:54.489661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-747d5827-56c8-47bc-914e-8d0d8c17914d None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:54.490887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-747d5827-56c8-47bc-914e-8d0d8c17914d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:54.494834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:54.598527 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 283/552] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:54.600109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d5827-56c8-47bc-914e-8d0d8c17914d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:54.607144 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:54.633046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e6c6d-0136-4b0c-8855-5844e9756506 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:54.681953 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbfc13-f130-49e8-8536-2d4187fea4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:54.683190 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbfc13-f130-49e8-8536-2d4187fea4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:54.683798 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbfc13-f130-49e8-8536-2d4187fea4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:54.684286 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbfc13-f130-49e8-8536-2d4187fea4d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:54.697270 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbfc13-f130-49e8-8536-2d4187fea4d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:54.706494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-dedbfc13-f130-49e8-8536-2d4187fea4d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:50:54.706494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:54.706494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:54.706494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:54.706494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:54.706494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:54.706494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:54.706494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:54.706494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:54.706494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:54.706494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:54.706494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:54.706494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:54.706494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:54.706494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:54.706494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:54.707668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:54.707668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:54.707668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:54.707668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:54.707668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:54.707668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:54.707668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:54.707668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:54.707668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:54.707668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:54.707668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:54.707668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:54.707668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:54.707668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:54.707668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:54.707668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:54.708494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:54.708494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:54.708494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:54.708494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:54.708494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:54.708494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:54.708494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:54.708494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:54.708494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:54.708494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:54.708494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:50:54.708494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:54.711535 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 270/553] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:50:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:54.712752 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbfc13-f130-49e8-8536-2d4187fea4d2 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:54.734275 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d5827-56c8-47bc-914e-8d0d8c17914d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:54.775848 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433260b-f03e-47f1-913d-aee0161c06fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:54.776571 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433260b-f03e-47f1-913d-aee0161c06fd None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:54.776880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433260b-f03e-47f1-913d-aee0161c06fd None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:54.777169 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433260b-f03e-47f1-913d-aee0161c06fd None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:54.778149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433260b-f03e-47f1-913d-aee0161c06fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:54.778991 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433260b-f03e-47f1-913d-aee0161c06fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:54.781588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433260b-f03e-47f1-913d-aee0161c06fd None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:54.798693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:54.826696 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 284/554] 10.222.0.76 () {56 vars in 1137 bytes} [Thu May 26 17:50:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:54.828613 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433260b-f03e-47f1-913d-aee0161c06fd None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:54.834649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbfc13-f130-49e8-8536-2d4187fea4d2 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:54.871468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6dd5ad-3a0b-4538-8e20-6f8e54c3ad1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:54.872671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6dd5ad-3a0b-4538-8e20-6f8e54c3ad1f None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:54.873245 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6dd5ad-3a0b-4538-8e20-6f8e54c3ad1f None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:54.873732 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6dd5ad-3a0b-4538-8e20-6f8e54c3ad1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:54.886172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6dd5ad-3a0b-4538-8e20-6f8e54c3ad1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:54.894695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-4e6dd5ad-3a0b-4538-8e20-6f8e54c3ad1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:50:54.894695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:54.894695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:54.894695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:54.894695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:54.894695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:54.894695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:54.894695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:54.894695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:54.894695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:54.894695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:54.894695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:54.894695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:54.894695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:54.894695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:54.894695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:54.895700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:54.895700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:54.895700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:54.895700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:54.895700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:54.895700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:54.895700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:54.895700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:54.895700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:54.895700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:54.895700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:54.895700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:54.895700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:54.895700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:54.895700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:54.895700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:54.896521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:54.896521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:54.896521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:54.896521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:54.896521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:54.896521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:54.896521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:54.896521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:54.896521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:54.896521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:54.896521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:50:54.896521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:54.899320 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 271/555] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:50:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:54.900360 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6dd5ad-3a0b-4538-8e20-6f8e54c3ad1f None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:54.937902 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:54.988372 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433260b-f03e-47f1-913d-aee0161c06fd None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:55.028360 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6dd5ad-3a0b-4538-8e20-6f8e54c3ad1f None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:55.032356 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c92cd-ec1a-4c71-81cd-40a2f5233797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:55.033037 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c92cd-ec1a-4c71-81cd-40a2f5233797 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:55.033317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c92cd-ec1a-4c71-81cd-40a2f5233797 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:55.033673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c92cd-ec1a-4c71-81cd-40a2f5233797 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:55.034664 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c92cd-ec1a-4c71-81cd-40a2f5233797 None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:55.035005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c92cd-ec1a-4c71-81cd-40a2f5233797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:55.036637 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c92cd-ec1a-4c71-81cd-40a2f5233797 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:55.045944 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 285/556] 10.222.0.76 () {56 vars in 1103 bytes} [Thu May 26 17:50:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:55.051782 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c92cd-ec1a-4c71-81cd-40a2f5233797 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:55.063826 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:55.071824 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ad249-6e03-4ba8-8123-f6dd7dd87d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:55.072721 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ad249-6e03-4ba8-8123-f6dd7dd87d64 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:55.073014 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ad249-6e03-4ba8-8123-f6dd7dd87d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:55.073301 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ad249-6e03-4ba8-8123-f6dd7dd87d64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:55.087084 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ad249-6e03-4ba8-8123-f6dd7dd87d64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:55.088619 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ad249-6e03-4ba8-8123-f6dd7dd87d64 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:55.109042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-069ad249-6e03-4ba8-8123-f6dd7dd87d64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:55.109042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:55.109042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:55.109042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:55.109042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:55.109042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:55.109042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:55.109042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:55.109042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:55.109042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:55.109042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:55.109042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:55.109042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:55.109042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:55.109042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:55.110675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:55.110675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:55.110675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:55.110675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:55.110675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:55.110675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:55.110675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:55.110675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:55.110675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:55.110675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:55.110675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:55.110675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:55.110675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:55.110675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:55.110675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:55.110675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:55.110675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:55.111715 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:55.111715 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:55.111715 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:55.111715 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:55.111715 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 272/557] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:55.112646 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffd657-7e7a-4bb2-848b-e40ec431be9b None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:55.112646 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffd657-7e7a-4bb2-848b-e40ec431be9b None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:55.113140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffd657-7e7a-4bb2-848b-e40ec431be9b None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:55.118096 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 273/558] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:55.118096 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffd657-7e7a-4bb2-848b-e40ec431be9b None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:55.186246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c92cd-ec1a-4c71-81cd-40a2f5233797 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:55.214946 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:55.230034 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109cbf1-9ef0-4312-8f8e-34eedeb49fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:55.230798 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109cbf1-9ef0-4312-8f8e-34eedeb49fdb None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:55.231087 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109cbf1-9ef0-4312-8f8e-34eedeb49fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:55.231656 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109cbf1-9ef0-4312-8f8e-34eedeb49fdb None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:55.232243 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109cbf1-9ef0-4312-8f8e-34eedeb49fdb None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:55.232716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109cbf1-9ef0-4312-8f8e-34eedeb49fdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:55.234388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109cbf1-9ef0-4312-8f8e-34eedeb49fdb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:55.236764 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffd657-7e7a-4bb2-848b-e40ec431be9b None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:55.249078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 286/559] 10.222.0.76 () {56 vars in 1103 bytes} [Thu May 26 17:50:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:55.249938 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109cbf1-9ef0-4312-8f8e-34eedeb49fdb None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:55.274751 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd700d-56dc-49dc-9dc5-0f9f1b2227c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:55.275567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd700d-56dc-49dc-9dc5-0f9f1b2227c4 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:55.275851 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd700d-56dc-49dc-9dc5-0f9f1b2227c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:55.276208 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd700d-56dc-49dc-9dc5-0f9f1b2227c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:55.288384 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd700d-56dc-49dc-9dc5-0f9f1b2227c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:55.289863 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd700d-56dc-49dc-9dc5-0f9f1b2227c4 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:55.309391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-16dd700d-56dc-49dc-9dc5-0f9f1b2227c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:55.309391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:55.309391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:55.309391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:55.309391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:55.309391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:55.309391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:55.309391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:55.309391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:55.309391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:55.309391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:55.309391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:55.309391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:55.309391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:55.309391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:55.310513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:55.310513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:55.310513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:55.310513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:55.310513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:55.310513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:55.310513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:55.310513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:55.310513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:55.310513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:55.310513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:55.310513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:55.310513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:55.310513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:55.310513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:55.310513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:55.310513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:55.311626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:55.311626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:55.311626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:55.311626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:55.315803 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 274/560] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:55.317180 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d3428-d524-42c2-9e97-dd328c9fc0f7 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:55.317480 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d3428-d524-42c2-9e97-dd328c9fc0f7 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:55.317768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d3428-d524-42c2-9e97-dd328c9fc0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:55.337675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-459d3428-d524-42c2-9e97-dd328c9fc0f7 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:55.338919 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-459d3428-d524-42c2-9e97-dd328c9fc0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:55.372405 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109cbf1-9ef0-4312-8f8e-34eedeb49fdb None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:55.372828 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:55.414872 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634f03c-ba23-4e07-a7e9-0cbccb969bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:55.415552 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634f03c-ba23-4e07-a7e9-0cbccb969bde None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:55.415901 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634f03c-ba23-4e07-a7e9-0cbccb969bde None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:55.416170 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634f03c-ba23-4e07-a7e9-0cbccb969bde None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:55.416826 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634f03c-ba23-4e07-a7e9-0cbccb969bde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:55.417131 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634f03c-ba23-4e07-a7e9-0cbccb969bde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:55.418566 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634f03c-ba23-4e07-a7e9-0cbccb969bde None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:55.419713 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 275/561] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:55.420394 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d3428-d524-42c2-9e97-dd328c9fc0f7 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:55.447275 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 287/562] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:55.448408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634f03c-ba23-4e07-a7e9-0cbccb969bde None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:55.506680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:55.566421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d3428-d524-42c2-9e97-dd328c9fc0f7 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:55.588306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634f03c-ba23-4e07-a7e9-0cbccb969bde None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:55.624040 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b4696-b391-4e38-8e10-ec5e1ed24cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:55.624968 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b4696-b391-4e38-8e10-ec5e1ed24cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:55.625259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b4696-b391-4e38-8e10-ec5e1ed24cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:55.625542 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b4696-b391-4e38-8e10-ec5e1ed24cf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:55.632465 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf45ad8-eb1d-48aa-8382-ab052eff3bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:55.633223 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf45ad8-eb1d-48aa-8382-ab052eff3bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:55.633544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf45ad8-eb1d-48aa-8382-ab052eff3bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:55.633827 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf45ad8-eb1d-48aa-8382-ab052eff3bf5 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:55.634582 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf45ad8-eb1d-48aa-8382-ab052eff3bf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:55.634891 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf45ad8-eb1d-48aa-8382-ab052eff3bf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:55.636396 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf45ad8-eb1d-48aa-8382-ab052eff3bf5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:55.638453 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b4696-b391-4e38-8e10-ec5e1ed24cf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:55.640349 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b4696-b391-4e38-8e10-ec5e1ed24cf8 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:55.654254 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 288/563] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:55.654254 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf45ad8-eb1d-48aa-8382-ab052eff3bf5 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:55.667809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-8b2b4696-b391-4e38-8e10-ec5e1ed24cf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:55.667809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:55.667809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:55.667809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:55.667809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:55.667809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:55.667809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:55.667809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:55.667809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:55.667809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:55.667809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:55.667809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:55.667809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:55.667809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:55.667809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:55.669008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:55.669008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:55.669008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:55.669008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:55.669008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:55.669008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:55.669008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:55.669008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:55.669008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:55.669008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:55.669008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:55.669008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:55.669008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:55.669008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:55.669008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:55.669008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:55.669008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:55.670086 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:55.670086 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:55.670086 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:55.670086 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:55.670086 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 276/564] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:55.670086 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b4696-b391-4e38-8e10-ec5e1ed24cf8 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:55.691911 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:55.765545 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf45ad8-eb1d-48aa-8382-ab052eff3bf5 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:55.767857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:55.779799 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b4696-b391-4e38-8e10-ec5e1ed24cf8 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:55.807083 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d7e62-71ec-4ec4-905b-6f9bd2774f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:55.807882 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d7e62-71ec-4ec4-905b-6f9bd2774f9c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:55.808158 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d7e62-71ec-4ec4-905b-6f9bd2774f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:55.808429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d7e62-71ec-4ec4-905b-6f9bd2774f9c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:55.817779 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4f7c7-0c4f-436f-9741-f79180c3fb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:55.818611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4f7c7-0c4f-436f-9741-f79180c3fb91 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:55.818888 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4f7c7-0c4f-436f-9741-f79180c3fb91 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:55.819165 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4f7c7-0c4f-436f-9741-f79180c3fb91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:55.819868 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d7e62-71ec-4ec4-905b-6f9bd2774f9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:55.826027 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4f7c7-0c4f-436f-9741-f79180c3fb91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:55.827514 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4f7c7-0c4f-436f-9741-f79180c3fb91 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:55.827652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-409d7e62-71ec-4ec4-905b-6f9bd2774f9c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:50:55.827652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:55.827652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:55.827652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:55.827652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:55.827652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:55.827652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:55.827652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:55.827652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:55.827652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:55.827652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:55.827652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:55.827652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:55.827652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:55.827652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:55.827652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:55.828617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:55.828617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:55.828617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:55.828617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:55.828617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:55.828617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:55.828617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:55.828617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:55.828617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:55.828617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:55.828617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:55.828617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:55.828617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:55.828617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:55.828617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:55.828617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:55.829446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:55.829446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:55.829446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:55.829446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:55.829446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:55.829446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:55.829446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:55.829446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:55.829446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:55.829446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:55.829446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:50:55.829446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:55.829446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 289/565] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:50:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:55.830396 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d7e62-71ec-4ec4-905b-6f9bd2774f9c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:55.844094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-7db4f7c7-0c4f-436f-9741-f79180c3fb91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:50:55.844094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:55.844094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:55.844094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:55.844094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:55.844094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:55.844094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:55.844094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:55.844094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:55.844094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:55.844094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:55.844094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:55.844094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:50:55.844094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:50:55.844094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:50:55.845424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:50:55.845424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:55.845424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:55.845424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:55.845424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:55.845424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:55.845424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:55.845424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:55.845424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:55.845424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:55.845424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:55.845424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:55.845424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:55.845424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:55.845424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:55.845424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:55.845424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:55.847513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:50:55.847513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:50:55.847513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:50:55.847513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:50:55.847513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:50:55.847513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:50:55.847513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:50:55.847513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:55.847513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 277/566] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:50:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:55.847513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4f7c7-0c4f-436f-9741-f79180c3fb91 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:55.927507 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:55.952735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d7e62-71ec-4ec4-905b-6f9bd2774f9c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:55.952981 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4f7c7-0c4f-436f-9741-f79180c3fb91 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:55.988707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207948d7-fade-4ceb-b326-acc1e2cf8800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:55.989824 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207948d7-fade-4ceb-b326-acc1e2cf8800 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:55.990068 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00def33-b2e9-485d-866c-b811332210e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:55.990277 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207948d7-fade-4ceb-b326-acc1e2cf8800 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:55.990622 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207948d7-fade-4ceb-b326-acc1e2cf8800 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:55.990761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00def33-b2e9-485d-866c-b811332210e6 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:55.991044 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00def33-b2e9-485d-866c-b811332210e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:55.991320 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00def33-b2e9-485d-866c-b811332210e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:55.992041 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00def33-b2e9-485d-866c-b811332210e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:55.992370 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00def33-b2e9-485d-866c-b811332210e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:55.993715 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00def33-b2e9-485d-866c-b811332210e6 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:55.999013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207948d7-fade-4ceb-b326-acc1e2cf8800 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:56.000459 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207948d7-fade-4ceb-b326-acc1e2cf8800 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:56.006519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 278/567] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:56.007915 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74551da-43f4-43ca-a915-68a4c027d96c None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:56.008207 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74551da-43f4-43ca-a915-68a4c027d96c None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:56.008480 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74551da-43f4-43ca-a915-68a4c027d96c None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:56.013766 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 279/568] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:56.013766 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74551da-43f4-43ca-a915-68a4c027d96c None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:56.015470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-207948d7-fade-4ceb-b326-acc1e2cf8800 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 17:50:56.015470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:56.015470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:56.015470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:56.015470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:56.015470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:56.015470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:56.015470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:56.015470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:56.015470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:56.015470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:56.015470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:56.015470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:50:56.015470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:50:56.015470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:50:56.015470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:50:56.017059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:56.017059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:56.017059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:50:56.017059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:50:56.017059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:50:56.017059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:50:56.017059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:56.017059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:56.017059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:56.017059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:56.017059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:56.017059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:56.017059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:56.017059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:56.017059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:56.017059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:56.017059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:56.017059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:56.018311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:56.018311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:56.018311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:50:56.018311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:50:56.018311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:50:56.018311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:50:56.018311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:50:56.018311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:50:56.018311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 17:50:56.018311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:56.018311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 290/569] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:50:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:56.018311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207948d7-fade-4ceb-b326-acc1e2cf8800 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:56.045099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:56.122660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207948d7-fade-4ceb-b326-acc1e2cf8800 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:56.126519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74551da-43f4-43ca-a915-68a4c027d96c None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:56.161754 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb13dec-f03a-4b9f-9367-e068096ea44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:56.162449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb13dec-f03a-4b9f-9367-e068096ea44b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:56.162725 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb13dec-f03a-4b9f-9367-e068096ea44b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:56.162998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb13dec-f03a-4b9f-9367-e068096ea44b None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:56.163782 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb13dec-f03a-4b9f-9367-e068096ea44b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:56.164087 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb13dec-f03a-4b9f-9367-e068096ea44b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:56.165550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb13dec-f03a-4b9f-9367-e068096ea44b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:56.167149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ec81a-fefa-4e33-8e4c-c23a208828b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:56.168006 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ec81a-fefa-4e33-8e4c-c23a208828b3 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:56.168725 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ec81a-fefa-4e33-8e4c-c23a208828b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:56.169098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ec81a-fefa-4e33-8e4c-c23a208828b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:56.169957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ec81a-fefa-4e33-8e4c-c23a208828b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:56.170395 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ec81a-fefa-4e33-8e4c-c23a208828b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:56.171840 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ec81a-fefa-4e33-8e4c-c23a208828b3 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:56.174164 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:56.183999 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 291/570] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:50:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:56.183999 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb13dec-f03a-4b9f-9367-e068096ea44b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:56.184282 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 280/571] 10.222.0.76 () {56 vars in 1103 bytes} [Thu May 26 17:50:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:56.184282 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccabc3-8ef5-42bb-bb7e-13fd6e983b58 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:56.184282 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccabc3-8ef5-42bb-bb7e-13fd6e983b58 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:56.184282 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccabc3-8ef5-42bb-bb7e-13fd6e983b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:56.201773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-fbccabc3-8ef5-42bb-bb7e-13fd6e983b58 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:56.203067 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-fbccabc3-8ef5-42bb-bb7e-13fd6e983b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:56.271426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:56.273742 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 281/572] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:56.280766 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbccabc3-8ef5-42bb-bb7e-13fd6e983b58 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:56.295707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb13dec-f03a-4b9f-9367-e068096ea44b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:56.332663 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7ff48-7b80-43ef-8fba-41af516468d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:56.333459 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7ff48-7b80-43ef-8fba-41af516468d3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:56.333748 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7ff48-7b80-43ef-8fba-41af516468d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:56.334019 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7ff48-7b80-43ef-8fba-41af516468d3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:56.341514 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa7ff48-7b80-43ef-8fba-41af516468d3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:56.343292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa7ff48-7b80-43ef-8fba-41af516468d3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:56.357972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-2fa7ff48-7b80-43ef-8fba-41af516468d3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 17:50:56.357972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:56.357972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:56.357972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:56.357972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:56.357972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:56.357972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:56.357972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:56.357972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:56.357972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:56.357972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:56.357972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:56.357972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:50:56.357972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:50:56.357972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:50:56.357972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:50:56.358983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:56.358983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:56.358983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:50:56.358983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:50:56.358983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:50:56.358983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:50:56.358983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:56.358983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:56.358983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:56.358983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:56.358983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:56.358983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:56.358983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:56.358983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:56.358983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:56.358983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:56.358983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:56.358983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:56.359943 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:56.359943 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:56.359943 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:50:56.359943 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:50:56.359943 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:50:56.359943 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:50:56.359943 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:50:56.359943 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:50:56.359943 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 17:50:56.359943 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:56.359943 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 292/573] 10.222.0.76 () {56 vars in 1093 bytes} [Thu May 26 17:50:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:56.359943 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7ff48-7b80-43ef-8fba-41af516468d3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:56.393195 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbccabc3-8ef5-42bb-bb7e-13fd6e983b58 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:56.421053 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:56.429994 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232ba27-502e-4ca1-9e9f-5bfe8af83068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:56.430705 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232ba27-502e-4ca1-9e9f-5bfe8af83068 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:56.430996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232ba27-502e-4ca1-9e9f-5bfe8af83068 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:56.431278 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232ba27-502e-4ca1-9e9f-5bfe8af83068 None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:56.432128 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0232ba27-502e-4ca1-9e9f-5bfe8af83068 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:56.432442 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0232ba27-502e-4ca1-9e9f-5bfe8af83068 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:56.433895 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0232ba27-502e-4ca1-9e9f-5bfe8af83068 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:56.453688 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 282/574] 10.222.0.76 () {56 vars in 1135 bytes} [Thu May 26 17:50:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:56.454559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232ba27-502e-4ca1-9e9f-5bfe8af83068 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:56.473635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7ff48-7b80-43ef-8fba-41af516468d3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:56.512701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64ad6b-a1a9-4338-84f0-0df413bfb40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:56.514131 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64ad6b-a1a9-4338-84f0-0df413bfb40d None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:56.514454 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64ad6b-a1a9-4338-84f0-0df413bfb40d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:56.515015 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64ad6b-a1a9-4338-84f0-0df413bfb40d None admin] PATH_INFO: `/v3/projects/ca1d09f491d944f99425a641561b03c3/users/52f4c9fc949c4a2c951b755234848cc7/roles/e776303775a2423aa0c62c161ecb5a7e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:56.530940 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f64ad6b-a1a9-4338-84f0-0df413bfb40d None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1d09f491d944f99425a641561b03c3, user_id=52f4c9fc949c4a2c951b755234848cc7, role_id=e776303775a2423aa0c62c161ecb5a7e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:56.532299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f64ad6b-a1a9-4338-84f0-0df413bfb40d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:56.534963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:56.547028 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 293/575] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:50:56 2022] PUT /identity/v3/projects/ca1d09f491d944f99425a641561b03c3/users/52f4c9fc949c4a2c951b755234848cc7/roles/e776303775a2423aa0c62c161ecb5a7e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:50:56.547951 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64ad6b-a1a9-4338-84f0-0df413bfb40d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:56.566235 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232ba27-502e-4ca1-9e9f-5bfe8af83068 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:56.606714 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e5d61-edc3-49cf-b4c4-7b97eaab0241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:56.607545 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e5d61-edc3-49cf-b4c4-7b97eaab0241 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:56.607832 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e5d61-edc3-49cf-b4c4-7b97eaab0241 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:56.608114 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e5d61-edc3-49cf-b4c4-7b97eaab0241 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:56.620689 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e5d61-edc3-49cf-b4c4-7b97eaab0241 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:56.622175 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e5d61-edc3-49cf-b4c4-7b97eaab0241 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:56.641904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-271e5d61-edc3-49cf-b4c4-7b97eaab0241 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:56.641904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:56.641904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:56.641904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:56.641904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:56.641904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:56.641904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:56.641904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:56.641904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:56.641904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:56.641904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:56.641904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:56.641904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:56.641904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:56.641904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:56.642993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:56.642993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:56.642993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:56.642993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:56.642993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:56.642993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:56.642993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:56.642993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:56.642993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:56.642993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:56.642993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:56.642993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:56.642993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:56.642993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:56.642993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:56.642993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:56.642993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:56.645248 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:56.645248 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:56.645248 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:56.645248 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:56.648520 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 283/576] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:56.649967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e5d61-edc3-49cf-b4c4-7b97eaab0241 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:56.687617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64ad6b-a1a9-4338-84f0-0df413bfb40d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:56.701364 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:56.727518 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de410e6c-e1cc-496a-9d22-4726f444ffa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:56.728349 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de410e6c-e1cc-496a-9d22-4726f444ffa4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:56.728634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de410e6c-e1cc-496a-9d22-4726f444ffa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:56.728908 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de410e6c-e1cc-496a-9d22-4726f444ffa4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:56.742979 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de410e6c-e1cc-496a-9d22-4726f444ffa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:56.751186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-de410e6c-e1cc-496a-9d22-4726f444ffa4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:50:56.751186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:56.751186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:56.751186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:56.751186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:56.751186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:56.751186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:56.751186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:56.751186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:56.751186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:56.751186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:56.751186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:56.751186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:56.751186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:56.751186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:56.751186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:56.752260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:56.752260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:56.752260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:56.752260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:56.752260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:56.752260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:56.752260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:56.752260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:56.752260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:56.752260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:56.752260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:56.752260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:56.752260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:56.752260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:56.752260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:56.752260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:56.754002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:56.754002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:56.754002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:56.754002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:56.754002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:56.754002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:56.754002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:56.754002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:56.754002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:56.754002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:56.754002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:50:56.754002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:56.754002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 294/577] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:50:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:56.755066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de410e6c-e1cc-496a-9d22-4726f444ffa4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:56.786933 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e5d61-edc3-49cf-b4c4-7b97eaab0241 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:56.825676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:56.833612 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4572be-b348-4d7d-9306-700ad5b52d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:56.834336 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4572be-b348-4d7d-9306-700ad5b52d47 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:56.834627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4572be-b348-4d7d-9306-700ad5b52d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:56.834905 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4572be-b348-4d7d-9306-700ad5b52d47 None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:56.835757 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4572be-b348-4d7d-9306-700ad5b52d47 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:56.836071 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4572be-b348-4d7d-9306-700ad5b52d47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:56.837518 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4572be-b348-4d7d-9306-700ad5b52d47 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:56.860028 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 284/578] 10.222.0.76 () {56 vars in 1133 bytes} [Thu May 26 17:50:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:56.861069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4572be-b348-4d7d-9306-700ad5b52d47 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:56.889075 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de410e6c-e1cc-496a-9d22-4726f444ffa4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:56.928377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd4f74-20c6-4aff-94f0-f503eb78289f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:56.929028 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd4f74-20c6-4aff-94f0-f503eb78289f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:56.929308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd4f74-20c6-4aff-94f0-f503eb78289f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:56.929584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd4f74-20c6-4aff-94f0-f503eb78289f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:56.969854 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:56.976395 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4572be-b348-4d7d-9306-700ad5b52d47 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:57.016087 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd4f74-20c6-4aff-94f0-f503eb78289f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:57.019437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95cccbb-d3b9-4792-ba86-f81609689856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:57.020115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95cccbb-d3b9-4792-ba86-f81609689856 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:57.020398 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95cccbb-d3b9-4792-ba86-f81609689856 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:57.020638 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd4f74-20c6-4aff-94f0-f503eb78289f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:57.020759 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95cccbb-d3b9-4792-ba86-f81609689856 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:57.021394 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95cccbb-d3b9-4792-ba86-f81609689856 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:57.021700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95cccbb-d3b9-4792-ba86-f81609689856 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:57.023095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95cccbb-d3b9-4792-ba86-f81609689856 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:57.036606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 285/579] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:57.037338 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95cccbb-d3b9-4792-ba86-f81609689856 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:57.095754 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:57.171862 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 295/580] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:50:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:50:57.171862 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd4f74-20c6-4aff-94f0-f503eb78289f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:57.175437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95cccbb-d3b9-4792-ba86-f81609689856 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:57.227143 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ce577-7713-4c1f-b828-8ae75d5c281f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:57.228210 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ce577-7713-4c1f-b828-8ae75d5c281f None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:57.228704 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ce577-7713-4c1f-b828-8ae75d5c281f None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:57.229182 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ce577-7713-4c1f-b828-8ae75d5c281f None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:57.230295 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ce577-7713-4c1f-b828-8ae75d5c281f None admin] RBAC: Adding query filter params (name=service) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:57.230845 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ce577-7713-4c1f-b828-8ae75d5c281f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:57.232502 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ce577-7713-4c1f-b828-8ae75d5c281f None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:57.243503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 286/581] 10.222.0.76 () {56 vars in 1103 bytes} [Thu May 26 17:50:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:57.244691 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ce577-7713-4c1f-b828-8ae75d5c281f None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:57.269252 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:57.289318 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd4f74-20c6-4aff-94f0-f503eb78289f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:57.329701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69232b91-76d9-4fb6-b719-3198dbfc1702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:57.331351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69232b91-76d9-4fb6-b719-3198dbfc1702 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:57.331780 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69232b91-76d9-4fb6-b719-3198dbfc1702 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:57.332368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69232b91-76d9-4fb6-b719-3198dbfc1702 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:57.344005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69232b91-76d9-4fb6-b719-3198dbfc1702 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:57.345480 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69232b91-76d9-4fb6-b719-3198dbfc1702 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:57.365957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-69232b91-76d9-4fb6-b719-3198dbfc1702 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:57.365957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:57.365957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:57.365957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:57.365957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:57.365957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:57.365957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:57.365957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:57.365957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:57.365957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:57.365957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:57.365957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:57.365957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:57.365957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:57.365957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:57.367126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:57.367126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:57.367126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:57.367126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:57.367126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:57.367126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:57.367126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:57.367126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:57.367126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:57.367126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:57.367126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:57.367126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:57.367126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:57.367126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:57.367126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:57.367126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:57.367126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:57.368375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:57.368375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:57.368375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:57.368375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:57.368375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 296/582] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:57.372392 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ce577-7713-4c1f-b828-8ae75d5c281f None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:57.372590 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69232b91-76d9-4fb6-b719-3198dbfc1702 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:57.382565 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:57.417470 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e59371-187e-4580-a42f-ba966959517f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:57.418635 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e59371-187e-4580-a42f-ba966959517f None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:57.419069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e59371-187e-4580-a42f-ba966959517f None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:57.419490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e59371-187e-4580-a42f-ba966959517f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:57.428355 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e59371-187e-4580-a42f-ba966959517f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:57.429956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e59371-187e-4580-a42f-ba966959517f None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:57.445531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-59e59371-187e-4580-a42f-ba966959517f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 17:50:57.445531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:57.445531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:57.445531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:57.445531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:57.445531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:57.445531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:57.445531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:57.445531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:57.445531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:57.445531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:57.445531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:57.445531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:50:57.445531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:50:57.445531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:50:57.445531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:50:57.446712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:57.446712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:57.446712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:50:57.446712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:50:57.446712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:50:57.446712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:50:57.446712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:57.446712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:57.446712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:57.446712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:57.446712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:57.446712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:57.446712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:57.446712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:57.446712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:57.446712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:57.446712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:57.446712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:57.447643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:57.447643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:57.447643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:50:57.447643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:50:57.447643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:50:57.447643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:50:57.447643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:50:57.447643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:50:57.447643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 17:50:57.447643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:57.447643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 287/583] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:50:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:57.447643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e59371-187e-4580-a42f-ba966959517f None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:57.500392 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69232b91-76d9-4fb6-b719-3198dbfc1702 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:57.537708 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca31450-0f30-4f09-b1f0-de6922bf0aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:57.538510 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca31450-0f30-4f09-b1f0-de6922bf0aef None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:57.538791 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca31450-0f30-4f09-b1f0-de6922bf0aef None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:57.539063 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca31450-0f30-4f09-b1f0-de6922bf0aef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:57.542657 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:57.551240 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca31450-0f30-4f09-b1f0-de6922bf0aef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:57.552694 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca31450-0f30-4f09-b1f0-de6922bf0aef None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:57.568676 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e59371-187e-4580-a42f-ba966959517f None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:57.570163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-bca31450-0f30-4f09-b1f0-de6922bf0aef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:57.570163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:57.570163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:57.570163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:57.570163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:57.570163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:57.570163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:57.570163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:57.570163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:57.570163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:57.570163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:57.570163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:57.570163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:57.570163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:57.570163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:57.571126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:57.571126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:57.571126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:57.571126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:57.571126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:57.571126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:57.571126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:57.571126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:57.571126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:57.571126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:57.571126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:57.571126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:57.571126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:57.571126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:57.571126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:57.571126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:57.571126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:57.571998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:57.571998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:57.571998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:57.571998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:57.576024 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 297/584] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:57.576678 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca31450-0f30-4f09-b1f0-de6922bf0aef None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:57.608251 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb6dae-251b-4f6c-81e6-349d5beaf9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:57.609054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb6dae-251b-4f6c-81e6-349d5beaf9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:57.609342 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb6dae-251b-4f6c-81e6-349d5beaf9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:57.609621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb6dae-251b-4f6c-81e6-349d5beaf9d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:57.621181 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb6dae-251b-4f6c-81e6-349d5beaf9d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:57.622690 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb6dae-251b-4f6c-81e6-349d5beaf9d3 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:57.642267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-a5bb6dae-251b-4f6c-81e6-349d5beaf9d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:57.642267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:57.642267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:57.642267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:57.642267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:57.642267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:57.642267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:57.642267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:57.642267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:57.642267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:57.642267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:57.642267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:57.642267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:57.642267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:57.642267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:57.643259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:57.643259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:57.643259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:57.643259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:57.643259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:57.643259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:57.643259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:57.643259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:57.643259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:57.643259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:57.643259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:57.643259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:57.643259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:57.643259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:57.643259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:57.643259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:57.643259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:57.644398 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:57.644398 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:57.644398 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:57.644398 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:57.644398 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 288/585] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:57.644398 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb6dae-251b-4f6c-81e6-349d5beaf9d3 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:57.645719 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:57.693757 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca31450-0f30-4f09-b1f0-de6922bf0aef None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:57.733831 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d551b0-a4d5-48c7-98f0-c6fb6440f978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:57.734536 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d551b0-a4d5-48c7-98f0-c6fb6440f978 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:57.734822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d551b0-a4d5-48c7-98f0-c6fb6440f978 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:57.735101 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d551b0-a4d5-48c7-98f0-c6fb6440f978 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:57.735841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d551b0-a4d5-48c7-98f0-c6fb6440f978 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:57.736135 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d551b0-a4d5-48c7-98f0-c6fb6440f978 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:57.738518 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d551b0-a4d5-48c7-98f0-c6fb6440f978 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:57.752669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 298/586] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:57.758707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d551b0-a4d5-48c7-98f0-c6fb6440f978 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:57.761475 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb6dae-251b-4f6c-81e6-349d5beaf9d3 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:57.796549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:57.804036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b3cc2-14db-4e1c-a95a-709b28a67405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:57.804730 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b3cc2-14db-4e1c-a95a-709b28a67405 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:57.805006 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b3cc2-14db-4e1c-a95a-709b28a67405 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:57.805287 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b3cc2-14db-4e1c-a95a-709b28a67405 None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:57.806234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b3cc2-14db-4e1c-a95a-709b28a67405 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:57.806501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b3cc2-14db-4e1c-a95a-709b28a67405 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:57.808130 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b3cc2-14db-4e1c-a95a-709b28a67405 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:57.827390 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 289/587] 10.222.0.76 () {56 vars in 1137 bytes} [Thu May 26 17:50:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:57.831773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b3cc2-14db-4e1c-a95a-709b28a67405 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:57.878221 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d551b0-a4d5-48c7-98f0-c6fb6440f978 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:57.890011 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:57.909677 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b71d7c-47b7-4eb6-a092-b500dcffd002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:57.910292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b71d7c-47b7-4eb6-a092-b500dcffd002 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:57.910581 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b71d7c-47b7-4eb6-a092-b500dcffd002 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:57.910862 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b71d7c-47b7-4eb6-a092-b500dcffd002 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:57.912056 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b71d7c-47b7-4eb6-a092-b500dcffd002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:57.912164 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b71d7c-47b7-4eb6-a092-b500dcffd002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:57.913541 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b71d7c-47b7-4eb6-a092-b500dcffd002 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:57.930365 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 299/588] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:57.930365 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b71d7c-47b7-4eb6-a092-b500dcffd002 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:57.944351 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b3cc2-14db-4e1c-a95a-709b28a67405 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:57.985960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebca47-b4a5-4090-9bdf-b280777d2512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:57.986618 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebca47-b4a5-4090-9bdf-b280777d2512 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:57.987585 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebca47-b4a5-4090-9bdf-b280777d2512 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:57.988051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebca47-b4a5-4090-9bdf-b280777d2512 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:57.989156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ebca47-b4a5-4090-9bdf-b280777d2512 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:57.989698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ebca47-b4a5-4090-9bdf-b280777d2512 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:57.992093 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ebca47-b4a5-4090-9bdf-b280777d2512 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:58.013168 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 290/589] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:58.013168 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebca47-b4a5-4090-9bdf-b280777d2512 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:58.062465 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:58.062648 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b71d7c-47b7-4eb6-a092-b500dcffd002 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:58.099662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b93481-9eed-4fab-8565-0b3e27281952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:58.100250 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b93481-9eed-4fab-8565-0b3e27281952 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:58.100535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b93481-9eed-4fab-8565-0b3e27281952 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:58.100863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b93481-9eed-4fab-8565-0b3e27281952 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:58.107919 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b93481-9eed-4fab-8565-0b3e27281952 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:58.109385 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b93481-9eed-4fab-8565-0b3e27281952 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:58.119223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebca47-b4a5-4090-9bdf-b280777d2512 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:58.125071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-02b93481-9eed-4fab-8565-0b3e27281952 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:50:58.125071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:58.125071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:58.125071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:58.125071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:58.125071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:58.125071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:58.125071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:58.125071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:58.125071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:58.125071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:58.125071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:58.125071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:50:58.125071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:50:58.125071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:50:58.126030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:50:58.126030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:58.126030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:58.126030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:58.126030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:58.126030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:58.126030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:58.126030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:58.126030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:58.126030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:58.126030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:58.126030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:58.126030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:58.126030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:58.126030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:58.126030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:58.126030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:58.126970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:50:58.126970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:50:58.126970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:50:58.126970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:50:58.126970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:50:58.126970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:50:58.126970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:50:58.126970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:58.126970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 300/590] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:50:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:58.127458 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b93481-9eed-4fab-8565-0b3e27281952 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:58.162293 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f52d2-3431-4789-8099-7e09a1d82c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:58.162941 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f52d2-3431-4789-8099-7e09a1d82c89 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:58.163231 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f52d2-3431-4789-8099-7e09a1d82c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:58.163515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f52d2-3431-4789-8099-7e09a1d82c89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:58.176108 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9f52d2-3431-4789-8099-7e09a1d82c89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:58.176643 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:58.177530 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9f52d2-3431-4789-8099-7e09a1d82c89 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:58.195675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-8c9f52d2-3431-4789-8099-7e09a1d82c89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:58.195675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:58.195675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:58.195675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:58.195675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:58.195675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:58.195675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:58.195675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:58.195675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:58.195675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:58.195675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:58.195675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:58.195675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:58.195675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:58.195675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:58.196645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:58.196645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:58.196645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:58.196645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:58.196645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:58.196645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:58.196645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:58.196645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:58.196645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:58.196645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:58.196645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:58.196645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:58.196645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:58.196645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:58.196645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:58.196645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:58.196645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:58.197561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:58.197561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:58.197561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:58.197561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:58.197561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 291/591] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:58.197830 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f52d2-3431-4789-8099-7e09a1d82c89 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:58.245359 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b93481-9eed-4fab-8565-0b3e27281952 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:58.288320 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff0cd0-eca7-44ba-a4df-d65af81ef48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:58.289401 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff0cd0-eca7-44ba-a4df-d65af81ef48b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:58.289763 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff0cd0-eca7-44ba-a4df-d65af81ef48b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:58.290016 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff0cd0-eca7-44ba-a4df-d65af81ef48b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:58.300930 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff0cd0-eca7-44ba-a4df-d65af81ef48b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:58.303131 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff0cd0-eca7-44ba-a4df-d65af81ef48b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:58.304797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:58.308329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f52d2-3431-4789-8099-7e09a1d82c89 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:58.319546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-9aff0cd0-eca7-44ba-a4df-d65af81ef48b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:50:58.319546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:58.319546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:58.319546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:58.319546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:58.319546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:58.319546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:58.319546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:58.319546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:58.319546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:58.319546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:58.319546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:58.319546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:50:58.319546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:50:58.319546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:50:58.320469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:50:58.320469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:58.320469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:58.320469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:58.320469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:58.320469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:58.320469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:58.320469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:58.320469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:58.320469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:58.320469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:58.320469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:58.320469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:58.320469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:58.320469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:58.320469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:58.320469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:58.321346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:50:58.321346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:50:58.321346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:50:58.321346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:50:58.321346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:50:58.321346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:50:58.321346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:50:58.321346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:58.321346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 301/592] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:50:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:58.322686 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff0cd0-eca7-44ba-a4df-d65af81ef48b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:58.351628 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eed6cb-a653-4651-a9ed-7322612aa252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:58.352754 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eed6cb-a653-4651-a9ed-7322612aa252 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:58.353181 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eed6cb-a653-4651-a9ed-7322612aa252 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:58.353455 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eed6cb-a653-4651-a9ed-7322612aa252 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:58.361570 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eed6cb-a653-4651-a9ed-7322612aa252 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:58.364222 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eed6cb-a653-4651-a9ed-7322612aa252 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:58.379409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-33eed6cb-a653-4651-a9ed-7322612aa252 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 17:50:58.379409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:58.379409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:58.379409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:58.379409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:58.379409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:58.379409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:58.379409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:58.379409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:58.379409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:58.379409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:58.379409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:58.379409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:50:58.379409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:50:58.379409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:50:58.379409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:50:58.380517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:58.380517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:58.380517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:50:58.380517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:50:58.380517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:50:58.380517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:50:58.380517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:58.380517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:58.380517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:58.380517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:58.380517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:58.380517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:58.380517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:58.380517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:58.380517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:58.380517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:58.380517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:58.380517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:58.381449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:58.381449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:58.381449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:50:58.381449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:50:58.381449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:50:58.381449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:50:58.381449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:50:58.381449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:50:58.381449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 17:50:58.381449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:58.381449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 292/593] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:50:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:58.382742 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eed6cb-a653-4651-a9ed-7322612aa252 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:58.433803 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:58.437343 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff0cd0-eca7-44ba-a4df-d65af81ef48b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:58.474697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018b9f6-cdb5-44c4-b17a-f66ca3bbc610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:58.475370 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018b9f6-cdb5-44c4-b17a-f66ca3bbc610 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:58.475652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018b9f6-cdb5-44c4-b17a-f66ca3bbc610 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:58.475927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018b9f6-cdb5-44c4-b17a-f66ca3bbc610 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:58.476709 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018b9f6-cdb5-44c4-b17a-f66ca3bbc610 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:58.477019 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018b9f6-cdb5-44c4-b17a-f66ca3bbc610 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:58.478468 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018b9f6-cdb5-44c4-b17a-f66ca3bbc610 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:58.492780 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 302/594] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:50:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:58.493617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018b9f6-cdb5-44c4-b17a-f66ca3bbc610 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:58.503566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eed6cb-a653-4651-a9ed-7322612aa252 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:58.548677 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8fad2-149a-4ddd-b7fd-0337777a80da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:58.549403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8fad2-149a-4ddd-b7fd-0337777a80da None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:58.550475 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8fad2-149a-4ddd-b7fd-0337777a80da None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:58.550835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8fad2-149a-4ddd-b7fd-0337777a80da None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:58.551559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8fad2-149a-4ddd-b7fd-0337777a80da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:58.551861 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8fad2-149a-4ddd-b7fd-0337777a80da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:58.553226 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8fad2-149a-4ddd-b7fd-0337777a80da None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:58.570563 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 293/595] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:58.570563 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8fad2-149a-4ddd-b7fd-0337777a80da None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:58.591783 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:58.603116 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018b9f6-cdb5-44c4-b17a-f66ca3bbc610 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:58.639707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfccd2-2387-4e77-a757-8660d43e6727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:58.640399 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfccd2-2387-4e77-a757-8660d43e6727 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:58.640684 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfccd2-2387-4e77-a757-8660d43e6727 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:58.640982 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfccd2-2387-4e77-a757-8660d43e6727 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:58.641773 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfccd2-2387-4e77-a757-8660d43e6727 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:58.642087 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfccd2-2387-4e77-a757-8660d43e6727 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:58.643908 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfccd2-2387-4e77-a757-8660d43e6727 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:58.660439 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 303/596] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:50:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:58.661824 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfccd2-2387-4e77-a757-8660d43e6727 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:58.685866 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:58.690773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8fad2-149a-4ddd-b7fd-0337777a80da None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:58.729916 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1ca71-4982-4a6e-8575-dc6bd8e047e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:58.730619 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1ca71-4982-4a6e-8575-dc6bd8e047e9 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:58.730908 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1ca71-4982-4a6e-8575-dc6bd8e047e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:58.731198 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1ca71-4982-4a6e-8575-dc6bd8e047e9 None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:58.732055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e1ca71-4982-4a6e-8575-dc6bd8e047e9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:58.732360 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e1ca71-4982-4a6e-8575-dc6bd8e047e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:58.733781 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e1ca71-4982-4a6e-8575-dc6bd8e047e9 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:58.753711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 294/597] 10.222.0.76 () {56 vars in 1139 bytes} [Thu May 26 17:50:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:58.754300 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1ca71-4982-4a6e-8575-dc6bd8e047e9 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:58.778276 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfccd2-2387-4e77-a757-8660d43e6727 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:58.826524 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d71ab9-6a3f-4d5e-bdf8-473ebfba0282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:58.827334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d71ab9-6a3f-4d5e-bdf8-473ebfba0282 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:58.827618 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d71ab9-6a3f-4d5e-bdf8-473ebfba0282 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:58.827893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d71ab9-6a3f-4d5e-bdf8-473ebfba0282 None admin] PATH_INFO: `/v3/projects/ca1d09f491d944f99425a641561b03c3/users/1899c317083d40f5b490cd9eb63fe521/roles/e776303775a2423aa0c62c161ecb5a7e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:58.841903 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:58.844457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d71ab9-6a3f-4d5e-bdf8-473ebfba0282 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1d09f491d944f99425a641561b03c3, user_id=1899c317083d40f5b490cd9eb63fe521, role_id=e776303775a2423aa0c62c161ecb5a7e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:58.845911 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d71ab9-6a3f-4d5e-bdf8-473ebfba0282 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:58.866429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 304/598] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:50:58 2022] PUT /identity/v3/projects/ca1d09f491d944f99425a641561b03c3/users/1899c317083d40f5b490cd9eb63fe521/roles/e776303775a2423aa0c62c161ecb5a7e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:50:58.866731 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1ca71-4982-4a6e-8575-dc6bd8e047e9 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:58.867698 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d71ab9-6a3f-4d5e-bdf8-473ebfba0282 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:58.913854 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec4aea-ec0c-4958-aabb-e9b147b9898d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:58.914725 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec4aea-ec0c-4958-aabb-e9b147b9898d None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:58.915012 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec4aea-ec0c-4958-aabb-e9b147b9898d None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:58.915288 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec4aea-ec0c-4958-aabb-e9b147b9898d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:58.923738 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfec4aea-ec0c-4958-aabb-e9b147b9898d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:58.925469 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfec4aea-ec0c-4958-aabb-e9b147b9898d None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:58.942410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-bfec4aea-ec0c-4958-aabb-e9b147b9898d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:50:58.942410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:58.942410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:58.942410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:58.942410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:58.942410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:58.942410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:58.942410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:58.942410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:58.942410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:58.942410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:58.942410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:58.942410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:50:58.942410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:50:58.942410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:50:58.943400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:50:58.943400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:58.943400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:58.943400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:58.943400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:58.943400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:58.943400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:58.943400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:58.943400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:58.943400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:58.943400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:58.943400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:58.943400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:58.943400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:58.943400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:58.943400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:58.943400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:58.944304 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:50:58.944304 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:50:58.944304 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:50:58.944304 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:50:58.944304 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:50:58.944304 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:50:58.944304 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:50:58.944304 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:58.946578 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 295/599] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:50:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:58.947656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec4aea-ec0c-4958-aabb-e9b147b9898d None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:58.950299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:58.991888 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d71ab9-6a3f-4d5e-bdf8-473ebfba0282 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:59.031588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb2d68-0a0f-43f8-8cec-c2c9deed182a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:59.032278 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb2d68-0a0f-43f8-8cec-c2c9deed182a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:59.032562 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb2d68-0a0f-43f8-8cec-c2c9deed182a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:59.032838 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb2d68-0a0f-43f8-8cec-c2c9deed182a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:59.033703 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb2d68-0a0f-43f8-8cec-c2c9deed182a None admin] RBAC: Adding query filter params (role.id=e776303775a2423aa0c62c161ecb5a7e, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=9ca774e864ca44cfb187e38bc8a8f6ea) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:59.034041 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb2d68-0a0f-43f8-8cec-c2c9deed182a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:59.035581 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb2d68-0a0f-43f8-8cec-c2c9deed182a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:59.042064 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 305/600] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:50:58 2022] GET /identity/v3/role_assignments?user.id=9ca774e864ca44cfb187e38bc8a8f6ea&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=e776303775a2423aa0c62c161ecb5a7e => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:59.043458 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a1cb7-c570-47e4-b49a-4162515bb4b8 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:59.043745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a1cb7-c570-47e4-b49a-4162515bb4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:59.044022 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a1cb7-c570-47e4-b49a-4162515bb4b8 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:59.049743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 306/601] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:50:59.049743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a1cb7-c570-47e4-b49a-4162515bb4b8 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:59.086296 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec4aea-ec0c-4958-aabb-e9b147b9898d None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:59.149318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27a93e-e81e-40ed-888e-47444858ab01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:59.150321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27a93e-e81e-40ed-888e-47444858ab01 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:59.150614 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27a93e-e81e-40ed-888e-47444858ab01 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:59.150898 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27a93e-e81e-40ed-888e-47444858ab01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:59.162646 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:59.173868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27a93e-e81e-40ed-888e-47444858ab01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:59.176305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27a93e-e81e-40ed-888e-47444858ab01 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:59.199778 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a1cb7-c570-47e4-b49a-4162515bb4b8 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:59.203707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-be27a93e-e81e-40ed-888e-47444858ab01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:59.203707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:59.203707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:59.203707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:59.203707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:59.203707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:59.203707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:59.203707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:59.203707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:59.203707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:59.203707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:59.203707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:59.203707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:50:59.203707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:50:59.203707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:50:59.204825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:50:59.204825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:59.204825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:59.204825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:59.204825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:59.204825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:59.204825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:59.204825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:59.204825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:59.204825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:59.204825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:59.204825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:59.204825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:59.204825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:59.204825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:59.204825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:59.204825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:59.206255 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:50:59.206255 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:50:59.206255 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:50:59.206255 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:59.208259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 296/602] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:59.209827 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e43a7b-c3d2-4c42-abe1-342fd1c25533 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:59.210367 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e43a7b-c3d2-4c42-abe1-342fd1c25533 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:59.210810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e43a7b-c3d2-4c42-abe1-342fd1c25533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:59.236301 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-53e43a7b-c3d2-4c42-abe1-342fd1c25533 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:50:59.238762 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-53e43a7b-c3d2-4c42-abe1-342fd1c25533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:50:59.240616 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:59.246966 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d01d81-aac5-4a0a-a8f0-be703d13fe30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:59.247888 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d01d81-aac5-4a0a-a8f0-be703d13fe30 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:59.248236 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d01d81-aac5-4a0a-a8f0-be703d13fe30 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:59.248579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d01d81-aac5-4a0a-a8f0-be703d13fe30 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:59.249447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d01d81-aac5-4a0a-a8f0-be703d13fe30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:59.249863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d01d81-aac5-4a0a-a8f0-be703d13fe30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:59.251542 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d01d81-aac5-4a0a-a8f0-be703d13fe30 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:59.265745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 307/603] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:50:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:59.266732 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d01d81-aac5-4a0a-a8f0-be703d13fe30 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:59.331876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 297/604] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:50:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:50:59.332941 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e43a7b-c3d2-4c42-abe1-342fd1c25533 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:59.387126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d01d81-aac5-4a0a-a8f0-be703d13fe30 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:59.425920 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e0420-6535-45a1-960c-4ccc5379dc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:59.426719 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e0420-6535-45a1-960c-4ccc5379dc78 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:59.427005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e0420-6535-45a1-960c-4ccc5379dc78 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:59.427282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e0420-6535-45a1-960c-4ccc5379dc78 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:59.428006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e0420-6535-45a1-960c-4ccc5379dc78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:59.428314 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e0420-6535-45a1-960c-4ccc5379dc78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:59.429882 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e0420-6535-45a1-960c-4ccc5379dc78 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:59.442788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 308/605] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:59.443727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e0420-6535-45a1-960c-4ccc5379dc78 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:59.454661 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e43a7b-c3d2-4c42-abe1-342fd1c25533 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:59.479220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:59.498892 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fab303-1981-4e38-92e6-521dd4562b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:59.499916 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fab303-1981-4e38-92e6-521dd4562b0a None admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:59.500196 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fab303-1981-4e38-92e6-521dd4562b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:59.500531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fab303-1981-4e38-92e6-521dd4562b0a None admin] PATH_INFO: `/v3/projects/ca1d09f491d944f99425a641561b03c3/users/76dbf8f3f1b44baca2d2aa90fe606a30/roles/e776303775a2423aa0c62c161ecb5a7e` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:59.502511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:59.520563 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fab303-1981-4e38-92e6-521dd4562b0a None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1d09f491d944f99425a641561b03c3, user_id=76dbf8f3f1b44baca2d2aa90fe606a30, role_id=e776303775a2423aa0c62c161ecb5a7e)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:59.522279 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fab303-1981-4e38-92e6-521dd4562b0a None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:59.546347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 298/606] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:50:59 2022] PUT /identity/v3/projects/ca1d09f491d944f99425a641561b03c3/users/76dbf8f3f1b44baca2d2aa90fe606a30/roles/e776303775a2423aa0c62c161ecb5a7e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:50:59.546347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fab303-1981-4e38-92e6-521dd4562b0a None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:59.612662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e0420-6535-45a1-960c-4ccc5379dc78 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:59.696637 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157201c-5159-4499-aefa-393a1242a1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:59.697691 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157201c-5159-4499-aefa-393a1242a1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:59.699533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157201c-5159-4499-aefa-393a1242a1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:59.700002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157201c-5159-4499-aefa-393a1242a1a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:59.723638 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9157201c-5159-4499-aefa-393a1242a1a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:50:59.736326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-9157201c-5159-4499-aefa-393a1242a1a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:50:59.736326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:59.736326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:59.736326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:59.736326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:59.736326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:59.736326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:59.736326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:59.736326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:59.736326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:59.736326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:59.736326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:59.736326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:50:59.736326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:50:59.736326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:50:59.736326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:50:59.737500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:50:59.737500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:50:59.737500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:50:59.737500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:50:59.737500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:59.737500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:59.737500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:59.737500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:59.737500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:59.737500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:59.737500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:59.737500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:50:59.737500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:59.737500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:59.737500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:59.737500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:59.738495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:59.738495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:59.738495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:59.738495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:59.738495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:50:59.738495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:50:59.738495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:50:59.738495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:50:59.738495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:50:59.738495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:50:59.738495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:50:59.738495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:50:59.745340 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 309/607] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:50:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:59.752083 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157201c-5159-4499-aefa-393a1242a1a6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:59.781599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fab303-1981-4e38-92e6-521dd4562b0a None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:59.822641 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6ddcd-a704-4fad-ae5a-6e9d5b082dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:59.823591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6ddcd-a704-4fad-ae5a-6e9d5b082dda None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:59.823927 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6ddcd-a704-4fad-ae5a-6e9d5b082dda None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:59.824206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6ddcd-a704-4fad-ae5a-6e9d5b082dda None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:59.832806 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6ddcd-a704-4fad-ae5a-6e9d5b082dda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:59.834296 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6ddcd-a704-4fad-ae5a-6e9d5b082dda None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:59.850403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-e1d6ddcd-a704-4fad-ae5a-6e9d5b082dda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:50:59.850403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:50:59.850403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:50:59.850403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:50:59.850403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:50:59.850403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:50:59.850403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:50:59.850403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:50:59.850403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:50:59.850403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:50:59.850403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:50:59.850403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:50:59.850403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:50:59.850403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:50:59.850403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:50:59.851534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:50:59.851534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:50:59.851534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:50:59.851534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:50:59.851534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:50:59.851534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:50:59.851534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:50:59.851534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:50:59.851534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:50:59.851534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:50:59.851534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:50:59.851534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:50:59.851534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:50:59.851534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:50:59.851534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:50:59.851534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:50:59.851534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:50:59.852691 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:50:59.852691 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:50:59.852691 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:50:59.852691 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:50:59.852691 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:50:59.852691 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:50:59.852691 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:50:59.852691 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:50:59.852691 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 299/608] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:50:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:50:59.857771 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6ddcd-a704-4fad-ae5a-6e9d5b082dda None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:59.876812 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157201c-5159-4499-aefa-393a1242a1a6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:50:59.892516 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:59.897548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:50:59.917169 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fa4bc-ccd8-41d8-9ce1-d87794d1d0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:50:59.918100 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fa4bc-ccd8-41d8-9ce1-d87794d1d0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:50:59.918560 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fa4bc-ccd8-41d8-9ce1-d87794d1d0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:50:59.919003 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fa4bc-ccd8-41d8-9ce1-d87794d1d0a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:50:59.919891 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4fa4bc-ccd8-41d8-9ce1-d87794d1d0a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:50:59.920339 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4fa4bc-ccd8-41d8-9ce1-d87794d1d0a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:50:59.921855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4fa4bc-ccd8-41d8-9ce1-d87794d1d0a2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:50:59.931093 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 310/609] 10.222.0.76 () {56 vars in 1103 bytes} [Thu May 26 17:50:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:50:59.937984 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fa4bc-ccd8-41d8-9ce1-d87794d1d0a2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:50:59.980202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6ddcd-a704-4fad-ae5a-6e9d5b082dda None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:00.021176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299175c-b500-48c8-ad35-a5706c78dc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:00.021812 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299175c-b500-48c8-ad35-a5706c78dc4a None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:00.022101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299175c-b500-48c8-ad35-a5706c78dc4a None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:00.022414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299175c-b500-48c8-ad35-a5706c78dc4a None admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:00.023222 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0299175c-b500-48c8-ad35-a5706c78dc4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:00.023543 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0299175c-b500-48c8-ad35-a5706c78dc4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:00.025406 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0299175c-b500-48c8-ad35-a5706c78dc4a None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:00.040422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 300/610] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:50:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:00.046417 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299175c-b500-48c8-ad35-a5706c78dc4a None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:00.131469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fa4bc-ccd8-41d8-9ce1-d87794d1d0a2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:00.159872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299175c-b500-48c8-ad35-a5706c78dc4a None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:00.170564 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:00.198219 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f51af0-eb09-4279-a3cb-9f2b1c9927ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:00.199121 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f51af0-eb09-4279-a3cb-9f2b1c9927ba None admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:00.199412 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f51af0-eb09-4279-a3cb-9f2b1c9927ba None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:00.199688 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f51af0-eb09-4279-a3cb-9f2b1c9927ba None admin] PATH_INFO: `/v3/projects/ca1d09f491d944f99425a641561b03c3/users/8df66990a3734209a350c8b6768d259f/roles/e776303775a2423aa0c62c161ecb5a7e` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:00.218771 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f51af0-eb09-4279-a3cb-9f2b1c9927ba None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1d09f491d944f99425a641561b03c3, user_id=8df66990a3734209a350c8b6768d259f, role_id=e776303775a2423aa0c62c161ecb5a7e)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:00.220356 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f51af0-eb09-4279-a3cb-9f2b1c9927ba None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:00.230113 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae856d-3e2d-41e0-84f9-f6b49b309317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:00.231040 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae856d-3e2d-41e0-84f9-f6b49b309317 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:00.231304 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae856d-3e2d-41e0-84f9-f6b49b309317 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:00.231575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae856d-3e2d-41e0-84f9-f6b49b309317 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:00.236071 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 301/611] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:51:00 2022] PUT /identity/v3/projects/ca1d09f491d944f99425a641561b03c3/users/8df66990a3734209a350c8b6768d259f/roles/e776303775a2423aa0c62c161ecb5a7e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 17:51:00.271592 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae856d-3e2d-41e0-84f9-f6b49b309317 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:00.273124 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae856d-3e2d-41e0-84f9-f6b49b309317 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:00.302416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-5cae856d-3e2d-41e0-84f9-f6b49b309317 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:00.302416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:00.302416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:00.302416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:00.302416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:00.302416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:00.302416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:00.302416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:00.302416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:00.302416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:00.302416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:00.302416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:00.302416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:00.302416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:00.302416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:00.303980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:00.303980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:00.303980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:00.303980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:00.303980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:00.303980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:00.303980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:00.303980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:00.303980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:00.303980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:00.303980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:00.303980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:00.303980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:00.303980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:00.303980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:00.303980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:00.303980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:00.305523 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:00.305523 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:00.305523 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:00.305523 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:00.310830 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 311/612] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:50:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:00.320316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f51af0-eb09-4279-a3cb-9f2b1c9927ba None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:00.470559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f51af0-eb09-4279-a3cb-9f2b1c9927ba None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:00.512263 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35c89d-324b-4098-98c6-0d5d4c8e1a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:00.512964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35c89d-324b-4098-98c6-0d5d4c8e1a7c None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:00.513251 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35c89d-324b-4098-98c6-0d5d4c8e1a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:00.513531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35c89d-324b-4098-98c6-0d5d4c8e1a7c None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:00.514382 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35c89d-324b-4098-98c6-0d5d4c8e1a7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:00.514695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35c89d-324b-4098-98c6-0d5d4c8e1a7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:00.516211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35c89d-324b-4098-98c6-0d5d4c8e1a7c None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:00.526736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:00.534456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 302/613] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:00.537720 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae856d-3e2d-41e0-84f9-f6b49b309317 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:00.569521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:00.653889 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae856d-3e2d-41e0-84f9-f6b49b309317 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:00.706572 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765211b-a689-4e65-bcee-7c9430fb7040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:00.707473 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765211b-a689-4e65-bcee-7c9430fb7040 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:00.707757 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765211b-a689-4e65-bcee-7c9430fb7040 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:00.708036 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765211b-a689-4e65-bcee-7c9430fb7040 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:00.720794 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765211b-a689-4e65-bcee-7c9430fb7040 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:00.723215 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765211b-a689-4e65-bcee-7c9430fb7040 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:00.744379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-9765211b-a689-4e65-bcee-7c9430fb7040 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 17:51:00.744379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:00.744379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:00.744379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:00.744379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:00.744379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:00.744379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:00.744379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:00.744379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:00.744379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:00.744379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:00.744379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:00.744379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:51:00.744379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:51:00.744379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:51:00.744379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:51:00.745380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:00.745380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:00.745380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:51:00.745380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:00.745380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:51:00.745380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:00.745380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:00.745380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:00.745380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:00.745380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:00.745380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:00.745380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:00.745380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:00.745380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:00.745380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:00.745380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:00.745380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:00.745380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:00.746338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:00.746338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:00.746338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:51:00.746338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:51:00.746338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:51:00.746338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:51:00.746338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:51:00.746338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:51:00.746338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 17:51:00.746338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:00.746876 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 312/614] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:51:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:00.753686 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35c89d-324b-4098-98c6-0d5d4c8e1a7c None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:00.911842 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35c89d-324b-4098-98c6-0d5d4c8e1a7c None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:00.963929 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b40b8e-b12d-45c0-997b-d4365f327914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:00.964608 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b40b8e-b12d-45c0-997b-d4365f327914 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:00.964892 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b40b8e-b12d-45c0-997b-d4365f327914 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:00.965168 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b40b8e-b12d-45c0-997b-d4365f327914 None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:00.966011 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b40b8e-b12d-45c0-997b-d4365f327914 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:00.966347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b40b8e-b12d-45c0-997b-d4365f327914 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:00.967792 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b40b8e-b12d-45c0-997b-d4365f327914 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:00.989808 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 303/615] 10.222.0.76 () {56 vars in 1137 bytes} [Thu May 26 17:51:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:00.999897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765211b-a689-4e65-bcee-7c9430fb7040 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:01.187725 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:01.217419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765211b-a689-4e65-bcee-7c9430fb7040 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:01.253764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9953ea5-deb4-4f53-9f78-c34dfd8b1760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:01.254478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9953ea5-deb4-4f53-9f78-c34dfd8b1760 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:01.254764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9953ea5-deb4-4f53-9f78-c34dfd8b1760 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:01.255040 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9953ea5-deb4-4f53-9f78-c34dfd8b1760 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:01.325351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9953ea5-deb4-4f53-9f78-c34dfd8b1760 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:01.326839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9953ea5-deb4-4f53-9f78-c34dfd8b1760 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:01.446705 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:01.475069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 313/616] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:51:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:51:01.480552 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b40b8e-b12d-45c0-997b-d4365f327914 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:01.589810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:01.620511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f34c2-660f-4225-ae65-1bcb2632898c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:01.621044 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f34c2-660f-4225-ae65-1bcb2632898c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:01.621417 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f34c2-660f-4225-ae65-1bcb2632898c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:01.625532 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 314/617] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:01.633586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91793524-3e40-43fc-91e4-0b75141a3c20 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:01.633876 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91793524-3e40-43fc-91e4-0b75141a3c20 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:01.634157 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91793524-3e40-43fc-91e4-0b75141a3c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:01.644762 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b40b8e-b12d-45c0-997b-d4365f327914 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:01.665718 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-91793524-3e40-43fc-91e4-0b75141a3c20 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:01.667394 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-91793524-3e40-43fc-91e4-0b75141a3c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:01.716094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f1968-f49e-466b-bd35-06a1df71c9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:01.717234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f1968-f49e-466b-bd35-06a1df71c9eb None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:01.717744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f1968-f49e-466b-bd35-06a1df71c9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:01.718270 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f1968-f49e-466b-bd35-06a1df71c9eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:01.734285 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f1968-f49e-466b-bd35-06a1df71c9eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:01.736029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f1968-f49e-466b-bd35-06a1df71c9eb None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:01.755092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-b38f1968-f49e-466b-bd35-06a1df71c9eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:01.755092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:01.755092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:01.755092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:01.755092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:01.755092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:01.755092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:01.755092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:01.755092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:01.755092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:01.755092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:01.755092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:01.755092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:01.755092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:01.755092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:01.756215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:01.756215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:01.756215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:01.756215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:01.756215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:01.756215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:01.756215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:01.756215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:01.756215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:01.756215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:01.756215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:01.756215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:01.756215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:01.756215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:01.756215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:01.756215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:01.756215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:01.757238 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:01.757238 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:01.757238 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:01.757238 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:01.759573 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 304/618] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:01.764609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f1968-f49e-466b-bd35-06a1df71c9eb None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:01.811424 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 315/619] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 17:51:01.855985 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91793524-3e40-43fc-91e4-0b75141a3c20 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:01.974570 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f1968-f49e-466b-bd35-06a1df71c9eb None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:01.997322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:02.004825 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91793524-3e40-43fc-91e4-0b75141a3c20 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:02.029015 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:02.031047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db402264-ff51-4bf1-8774-9e727e206995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:02.032150 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db402264-ff51-4bf1-8774-9e727e206995 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:02.032917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db402264-ff51-4bf1-8774-9e727e206995 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:02.033345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db402264-ff51-4bf1-8774-9e727e206995 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:02.035449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db402264-ff51-4bf1-8774-9e727e206995 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:02.035938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db402264-ff51-4bf1-8774-9e727e206995 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:02.037852 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db402264-ff51-4bf1-8774-9e727e206995 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:02.051642 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 305/620] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:02.052451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f4108-e79a-45e7-8e79-2afc8412f5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:02.053363 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f4108-e79a-45e7-8e79-2afc8412f5ad None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:02.053697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3e1c2-ef37-428f-a9e6-ce25a55957bf None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:02.053841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f4108-e79a-45e7-8e79-2afc8412f5ad None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:02.054280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f4108-e79a-45e7-8e79-2afc8412f5ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:02.054450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3e1c2-ef37-428f-a9e6-ce25a55957bf None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:02.054865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3e1c2-ef37-428f-a9e6-ce25a55957bf None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:02.056777 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 306/621] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:02.057846 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3e1c2-ef37-428f-a9e6-ce25a55957bf None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:02.073074 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f4108-e79a-45e7-8e79-2afc8412f5ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:51:02.085488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-9e7f4108-e79a-45e7-8e79-2afc8412f5ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:51:02.085488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:02.085488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:02.085488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:02.085488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:02.085488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:02.085488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:02.085488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:02.085488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:02.085488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:02.085488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:02.085488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:02.085488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:51:02.085488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:51:02.085488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:51:02.085488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:51:02.086551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:51:02.086551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:51:02.086551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:51:02.086551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:51:02.086551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:02.086551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:02.086551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:02.086551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:02.086551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:02.086551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:02.086551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:02.086551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:02.086551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:02.086551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:02.086551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:02.086551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:02.087704 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:02.087704 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:02.087704 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:02.087704 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:02.087704 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:51:02.087704 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:51:02.087704 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:51:02.087704 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:51:02.087704 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:51:02.087704 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:51:02.087704 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:51:02.087704 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:02.087704 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 316/622] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:51:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:51:02.090823 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228891a2-2525-4cf0-b7d4-da0cb25e9322 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:02.091334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228891a2-2525-4cf0-b7d4-da0cb25e9322 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:02.091821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228891a2-2525-4cf0-b7d4-da0cb25e9322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:02.109788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-228891a2-2525-4cf0-b7d4-da0cb25e9322 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:02.111394 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-228891a2-2525-4cf0-b7d4-da0cb25e9322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:02.193418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 317/623] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:51:02.194763 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228891a2-2525-4cf0-b7d4-da0cb25e9322 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:02.202930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3e1c2-ef37-428f-a9e6-ce25a55957bf None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:02.251741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415da55-1069-462e-a62c-8422df503c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:02.253221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415da55-1069-462e-a62c-8422df503c57 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:02.253514 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415da55-1069-462e-a62c-8422df503c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:02.253797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415da55-1069-462e-a62c-8422df503c57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:02.263827 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415da55-1069-462e-a62c-8422df503c57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:02.265898 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415da55-1069-462e-a62c-8422df503c57 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:02.281171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-2415da55-1069-462e-a62c-8422df503c57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:02.281171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:02.281171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:02.281171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:02.281171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:02.281171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:02.281171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:02.281171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:02.281171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:02.281171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:02.281171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:02.281171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:02.281171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:51:02.281171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:51:02.281171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:51:02.282172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:51:02.282172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:02.282172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:02.282172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:02.282172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:02.282172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:02.282172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:02.282172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:02.282172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:02.282172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:02.282172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:02.282172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:02.282172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:02.282172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:02.282172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:02.282172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:02.282172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:02.283100 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:51:02.283100 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:51:02.283100 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:51:02.283100 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:51:02.283100 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:51:02.283100 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:51:02.283100 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:02.283100 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:02.283100 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 307/624] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:02.284626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415da55-1069-462e-a62c-8422df503c57 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:02.320322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:02.328601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228891a2-2525-4cf0-b7d4-da0cb25e9322 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:02.339274 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:02.378027 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168641e4-c84c-4978-a579-530056739b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:02.379174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168641e4-c84c-4978-a579-530056739b15 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:02.379568 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168641e4-c84c-4978-a579-530056739b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:02.379919 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168641e4-c84c-4978-a579-530056739b15 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:02.381357 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168641e4-c84c-4978-a579-530056739b15 None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:02.381357 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168641e4-c84c-4978-a579-530056739b15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:02.382701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168641e4-c84c-4978-a579-530056739b15 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:02.398067 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 318/625] 10.222.0.76 () {56 vars in 1103 bytes} [Thu May 26 17:51:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:02.398067 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168641e4-c84c-4978-a579-530056739b15 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:02.423845 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415da55-1069-462e-a62c-8422df503c57 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:02.470685 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d655dbf-1176-4305-ba9a-4d2fe1d9f760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:02.472737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d655dbf-1176-4305-ba9a-4d2fe1d9f760 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:02.473359 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d655dbf-1176-4305-ba9a-4d2fe1d9f760 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:02.473914 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d655dbf-1176-4305-ba9a-4d2fe1d9f760 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:02.487982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d655dbf-1176-4305-ba9a-4d2fe1d9f760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:51:02.496609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-4d655dbf-1176-4305-ba9a-4d2fe1d9f760 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:51:02.496609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:02.496609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:02.496609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:02.496609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:02.496609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:02.496609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:02.496609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:02.496609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:02.496609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:02.496609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:02.496609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:02.496609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:51:02.496609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:51:02.496609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:51:02.496609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:51:02.497890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:51:02.497890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:51:02.497890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:51:02.497890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:51:02.497890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:02.497890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:02.497890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:02.497890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:02.497890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:02.497890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:02.497890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:02.497890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:02.497890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:02.497890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:02.497890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:02.497890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:02.498950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:02.498950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:02.498950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:02.498950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:02.498950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:51:02.498950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:51:02.498950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:51:02.498950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:51:02.498950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:51:02.498950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:51:02.498950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:51:02.498950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:02.498950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 308/626] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:51:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:02.498950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d655dbf-1176-4305-ba9a-4d2fe1d9f760 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:02.522434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168641e4-c84c-4978-a579-530056739b15 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:02.567335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcd8a2-30aa-4f59-b152-eaed26b5246e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:02.568272 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcd8a2-30aa-4f59-b152-eaed26b5246e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:02.568559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcd8a2-30aa-4f59-b152-eaed26b5246e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:02.569449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcd8a2-30aa-4f59-b152-eaed26b5246e None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:02.570431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcd8a2-30aa-4f59-b152-eaed26b5246e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:02.570750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcd8a2-30aa-4f59-b152-eaed26b5246e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:02.572253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcd8a2-30aa-4f59-b152-eaed26b5246e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:02.587941 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 319/627] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:51:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:02.588730 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b63512-6881-4b8f-afa8-150c3a072358 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:02.589022 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b63512-6881-4b8f-afa8-150c3a072358 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:02.589287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b63512-6881-4b8f-afa8-150c3a072358 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:02.590785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 320/628] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:02.595419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b63512-6881-4b8f-afa8-150c3a072358 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:02.606532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:02.620168 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d655dbf-1176-4305-ba9a-4d2fe1d9f760 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:02.628284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:02.664752 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e1a5d-e995-46eb-be7c-9f75d61dfd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:02.665868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e1a5d-e995-46eb-be7c-9f75d61dfd4a None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:02.666361 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e1a5d-e995-46eb-be7c-9f75d61dfd4a None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:02.666817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e1a5d-e995-46eb-be7c-9f75d61dfd4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:02.678546 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e1a5d-e995-46eb-be7c-9f75d61dfd4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:02.680013 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e1a5d-e995-46eb-be7c-9f75d61dfd4a None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:02.698117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-e91e1a5d-e995-46eb-be7c-9f75d61dfd4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:02.698117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:02.698117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:02.698117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:02.698117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:02.698117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:02.698117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:02.698117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:02.698117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:02.698117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:02.698117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:02.698117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:02.698117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:02.698117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:02.698117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:02.699254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:02.699254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:02.699254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:02.699254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:02.699254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:02.699254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:02.699254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:02.699254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:02.699254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:02.699254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:02.699254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:02.699254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:02.699254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:02.699254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:02.699254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:02.699254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:02.699254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:02.700129 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:02.700129 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:02.700129 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:02.700129 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:02.700129 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 309/629] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:02.700411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e1a5d-e995-46eb-be7c-9f75d61dfd4a None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:02.716899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b63512-6881-4b8f-afa8-150c3a072358 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:02.759940 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfef3c5-1902-4b7a-8aa7-261d84b204fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:02.760617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfef3c5-1902-4b7a-8aa7-261d84b204fe None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:02.761484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfef3c5-1902-4b7a-8aa7-261d84b204fe None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:02.761767 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfef3c5-1902-4b7a-8aa7-261d84b204fe None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:02.762564 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfef3c5-1902-4b7a-8aa7-261d84b204fe None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:02.762870 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfef3c5-1902-4b7a-8aa7-261d84b204fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:02.764249 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfef3c5-1902-4b7a-8aa7-261d84b204fe None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:02.776856 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 321/630] 10.222.0.76 () {56 vars in 1103 bytes} [Thu May 26 17:51:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:02.776856 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9888ae3-c705-4ea5-af12-eaf51a0e2536 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:02.776856 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9888ae3-c705-4ea5-af12-eaf51a0e2536 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:02.776856 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9888ae3-c705-4ea5-af12-eaf51a0e2536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:02.798112 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d9888ae3-c705-4ea5-af12-eaf51a0e2536 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:02.799662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d9888ae3-c705-4ea5-af12-eaf51a0e2536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:02.823910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e1a5d-e995-46eb-be7c-9f75d61dfd4a None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:02.863754 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d33c5-b9b8-4ca2-b3be-b3109a669449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:02.864415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d33c5-b9b8-4ca2-b3be-b3109a669449 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:02.864698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d33c5-b9b8-4ca2-b3be-b3109a669449 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:02.864973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d33c5-b9b8-4ca2-b3be-b3109a669449 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:02.865797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d33c5-b9b8-4ca2-b3be-b3109a669449 None admin] RBAC: Adding query filter params (role.id=e776303775a2423aa0c62c161ecb5a7e, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=52f4c9fc949c4a2c951b755234848cc7) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:02.866111 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d33c5-b9b8-4ca2-b3be-b3109a669449 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:02.867592 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d33c5-b9b8-4ca2-b3be-b3109a669449 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:02.874291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 310/631] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:51:02 2022] GET /identity/v3/role_assignments?user.id=52f4c9fc949c4a2c951b755234848cc7&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=e776303775a2423aa0c62c161ecb5a7e => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:02.875101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d33c5-b9b8-4ca2-b3be-b3109a669449 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:02.891437 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 322/632] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:51:02.892868 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9888ae3-c705-4ea5-af12-eaf51a0e2536 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:02.894690 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:02.934970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:03.034831 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d33c5-b9b8-4ca2-b3be-b3109a669449 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:03.060314 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9888ae3-c705-4ea5-af12-eaf51a0e2536 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:03.086041 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb079f-08ae-449d-8fef-94f50931cf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:03.086788 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb079f-08ae-449d-8fef-94f50931cf2c None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:03.087075 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb079f-08ae-449d-8fef-94f50931cf2c None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:03.087353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb079f-08ae-449d-8fef-94f50931cf2c None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:03.088313 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb079f-08ae-449d-8fef-94f50931cf2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:03.088432 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb079f-08ae-449d-8fef-94f50931cf2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:03.089847 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb079f-08ae-449d-8fef-94f50931cf2c None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:03.108605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 311/633] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:03.109611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb079f-08ae-449d-8fef-94f50931cf2c None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:03.115288 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486217e9-0f6d-426b-a5a2-1fbe247ce908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:03.116103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486217e9-0f6d-426b-a5a2-1fbe247ce908 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:03.116382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486217e9-0f6d-426b-a5a2-1fbe247ce908 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:03.116660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486217e9-0f6d-426b-a5a2-1fbe247ce908 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:03.135448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486217e9-0f6d-426b-a5a2-1fbe247ce908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:03.136937 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486217e9-0f6d-426b-a5a2-1fbe247ce908 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:03.159273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-486217e9-0f6d-426b-a5a2-1fbe247ce908 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:03.159273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:03.159273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:03.159273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:03.159273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:03.159273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:03.159273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:03.159273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:03.159273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:03.159273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:03.159273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:03.159273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:03.159273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:03.159273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:03.159273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:03.160346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:03.160346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:03.160346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:03.160346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:03.160346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:03.160346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:03.160346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:03.160346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:03.160346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:03.160346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:03.160346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:03.160346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:03.160346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:03.160346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:03.160346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:03.160346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:03.160346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:03.161529 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:03.161529 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:03.161529 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:03.161529 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:03.161529 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 323/634] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:03.162367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6f448-f3cc-479e-a3a8-cdd8dffa0755 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:03.162467 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6f448-f3cc-479e-a3a8-cdd8dffa0755 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:03.162683 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6f448-f3cc-479e-a3a8-cdd8dffa0755 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:03.164121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 324/635] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:03.174092 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6f448-f3cc-479e-a3a8-cdd8dffa0755 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:03.200525 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:03.241711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb079f-08ae-449d-8fef-94f50931cf2c None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:03.264683 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:03.282620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566e3ed-7d43-43f9-b1bc-811aa9583ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:03.283436 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566e3ed-7d43-43f9-b1bc-811aa9583ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:03.283807 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566e3ed-7d43-43f9-b1bc-811aa9583ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:03.284448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566e3ed-7d43-43f9-b1bc-811aa9583ca7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:03.296163 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566e3ed-7d43-43f9-b1bc-811aa9583ca7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:51:03.298350 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6f448-f3cc-479e-a3a8-cdd8dffa0755 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:03.304496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-7566e3ed-7d43-43f9-b1bc-811aa9583ca7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:51:03.304496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:03.304496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:03.304496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:03.304496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:03.304496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:03.304496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:03.304496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:03.304496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:03.304496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:03.304496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:03.304496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:03.304496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:51:03.304496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:51:03.304496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:51:03.304496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:51:03.305534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:51:03.305534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:51:03.305534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:51:03.305534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:51:03.305534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:03.305534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:03.305534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:03.305534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:03.305534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:03.305534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:03.305534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:03.305534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:03.305534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:03.305534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:03.305534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:03.305534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:03.306422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:03.306422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:03.306422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:03.306422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:03.306422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:51:03.306422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:51:03.306422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:51:03.306422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:51:03.306422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:51:03.306422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:51:03.306422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:51:03.306422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:03.306422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 312/636] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:51:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:03.307599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa2c72-ad94-4431-a989-6c0609b4a2d9 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:03.307599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa2c72-ad94-4431-a989-6c0609b4a2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:03.307891 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa2c72-ad94-4431-a989-6c0609b4a2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:03.325808 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-bdaa2c72-ad94-4431-a989-6c0609b4a2d9 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:03.327052 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-bdaa2c72-ad94-4431-a989-6c0609b4a2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:03.341952 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff7e50-88f5-48c7-9f61-ddf0d76a48af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:03.342788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff7e50-88f5-48c7-9f61-ddf0d76a48af None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:03.343075 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff7e50-88f5-48c7-9f61-ddf0d76a48af None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:03.343358 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff7e50-88f5-48c7-9f61-ddf0d76a48af None admin] PATH_INFO: `/v3/users/nova` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:03.351432 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff7e50-88f5-48c7-9f61-ddf0d76a48af None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:03.352790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff7e50-88f5-48c7-9f61-ddf0d76a48af None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:03.368247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-d0ff7e50-88f5-48c7-9f61-ddf0d76a48af None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 17:51:03.368247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:03.368247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:03.368247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:03.368247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:03.368247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:03.368247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:03.368247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:03.368247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:03.368247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:03.368247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:03.368247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:03.368247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:51:03.368247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:51:03.368247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:51:03.368247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:51:03.369313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:03.369313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:03.369313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:51:03.369313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:03.369313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:51:03.369313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:03.369313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:03.369313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:03.369313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:03.369313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:03.369313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:03.369313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:03.369313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:03.369313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:03.369313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:03.369313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:03.369313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:03.369313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:03.370322 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:03.370322 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:03.370322 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:51:03.370322 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:51:03.370322 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:51:03.370322 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:51:03.370322 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:51:03.370322 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:51:03.370322 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 17:51:03.370322 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:03.374312 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 325/637] 10.222.0.76 () {56 vars in 1093 bytes} [Thu May 26 17:51:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:03.374937 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff7e50-88f5-48c7-9f61-ddf0d76a48af None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:03.410250 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 313/638] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:51:03.410250 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa2c72-ad94-4431-a989-6c0609b4a2d9 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:03.472106 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:03.503844 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff7e50-88f5-48c7-9f61-ddf0d76a48af None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:03.521678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa2c72-ad94-4431-a989-6c0609b4a2d9 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:03.537549 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:03.544784 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf9564-04e8-4a4e-aa14-54a855317a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:03.545737 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf9564-04e8-4a4e-aa14-54a855317a5c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:03.546028 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf9564-04e8-4a4e-aa14-54a855317a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:03.546379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf9564-04e8-4a4e-aa14-54a855317a5c None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:03.547345 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf9564-04e8-4a4e-aa14-54a855317a5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:03.548056 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf9564-04e8-4a4e-aa14-54a855317a5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:03.550033 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf9564-04e8-4a4e-aa14-54a855317a5c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:03.559031 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6cda62-6e16-4f62-ac5d-e0f5d6aafaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:03.560162 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cda62-6e16-4f62-ac5d-e0f5d6aafaba None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:03.560455 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cda62-6e16-4f62-ac5d-e0f5d6aafaba None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:03.561454 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cda62-6e16-4f62-ac5d-e0f5d6aafaba None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:03.562618 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6cda62-6e16-4f62-ac5d-e0f5d6aafaba None admin] RBAC: Adding query filter params (name=service) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:03.562931 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6cda62-6e16-4f62-ac5d-e0f5d6aafaba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:03.564319 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6cda62-6e16-4f62-ac5d-e0f5d6aafaba None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:03.568118 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 326/639] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:03.569038 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf9564-04e8-4a4e-aa14-54a855317a5c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:03.574352 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 314/640] 10.222.0.76 () {56 vars in 1103 bytes} [Thu May 26 17:51:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:03.575055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6cda62-6e16-4f62-ac5d-e0f5d6aafaba None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:03.684094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6cda62-6e16-4f62-ac5d-e0f5d6aafaba None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:03.684894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf9564-04e8-4a4e-aa14-54a855317a5c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:03.703888 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:03.722112 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04ed06-bcf2-4e6a-8225-54310db0f785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:03.722848 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04ed06-bcf2-4e6a-8225-54310db0f785 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:03.723130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04ed06-bcf2-4e6a-8225-54310db0f785 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:03.723413 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04ed06-bcf2-4e6a-8225-54310db0f785 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:03.723719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410eb787-a7d5-4048-8f50-a329e9c5427b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:03.724252 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04ed06-bcf2-4e6a-8225-54310db0f785 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:03.724526 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410eb787-a7d5-4048-8f50-a329e9c5427b None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:03.724620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04ed06-bcf2-4e6a-8225-54310db0f785 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:03.724810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410eb787-a7d5-4048-8f50-a329e9c5427b None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:03.725082 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410eb787-a7d5-4048-8f50-a329e9c5427b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:03.726014 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04ed06-bcf2-4e6a-8225-54310db0f785 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:03.738411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410eb787-a7d5-4048-8f50-a329e9c5427b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:51:03.749360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 327/641] 10.222.0.76 () {56 vars in 1133 bytes} [Thu May 26 17:51:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:03.749360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04ed06-bcf2-4e6a-8225-54310db0f785 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:03.749637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-410eb787-a7d5-4048-8f50-a329e9c5427b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:51:03.749637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:03.749637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:03.749637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:03.749637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:03.749637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:03.749637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:03.749637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:03.749637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:03.749637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:03.749637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:03.749637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:03.749637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:51:03.749637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:51:03.749637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:51:03.749637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:51:03.750549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:51:03.750549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:51:03.750549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:51:03.750549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:51:03.750549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:03.750549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:03.750549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:03.750549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:03.750549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:03.750549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:03.750549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:03.750549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:03.750549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:03.750549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:03.750549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:03.750549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:03.751411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:03.751411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:03.751411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:03.751411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:03.751411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:51:03.751411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:51:03.751411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:51:03.751411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:51:03.751411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:51:03.751411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:51:03.751411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:51:03.751411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:03.751411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 315/642] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:51:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:03.751411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410eb787-a7d5-4048-8f50-a329e9c5427b None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:03.809698 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:03.863847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04ed06-bcf2-4e6a-8225-54310db0f785 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:03.870333 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410eb787-a7d5-4048-8f50-a329e9c5427b None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:03.901127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82608c18-cc06-4c6d-b6b0-1a51c8499d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:03.901923 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82608c18-cc06-4c6d-b6b0-1a51c8499d8a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:03.902234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82608c18-cc06-4c6d-b6b0-1a51c8499d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:03.902520 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82608c18-cc06-4c6d-b6b0-1a51c8499d8a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:03.907977 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebeadf6-ee8b-4c20-adb2-28ffacae5120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:03.908849 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebeadf6-ee8b-4c20-adb2-28ffacae5120 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:03.909986 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebeadf6-ee8b-4c20-adb2-28ffacae5120 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:03.910345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebeadf6-ee8b-4c20-adb2-28ffacae5120 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:03.910523 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82608c18-cc06-4c6d-b6b0-1a51c8499d8a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:03.912221 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82608c18-cc06-4c6d-b6b0-1a51c8499d8a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:03.922606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebeadf6-ee8b-4c20-adb2-28ffacae5120 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:03.924069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebeadf6-ee8b-4c20-adb2-28ffacae5120 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:03.927121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-82608c18-cc06-4c6d-b6b0-1a51c8499d8a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 17:51:03.927121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:03.927121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:03.927121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:03.927121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:03.927121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:03.927121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:03.927121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:03.927121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:03.927121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:03.927121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:03.927121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:03.927121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:51:03.927121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:51:03.927121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:51:03.927121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:51:03.928130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:03.928130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:03.928130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:51:03.928130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:03.928130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:51:03.928130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:03.928130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:03.928130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:03.928130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:03.928130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:03.928130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:03.928130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:03.928130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:03.928130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:03.928130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:03.928130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:03.928130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:03.928130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:03.934453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:03.934453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:03.934453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:51:03.934453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:51:03.934453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:51:03.934453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:51:03.934453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:51:03.934453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:51:03.934453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 17:51:03.934453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:03.934453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 328/643] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:51:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:03.934453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82608c18-cc06-4c6d-b6b0-1a51c8499d8a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:03.945098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-7ebeadf6-ee8b-4c20-adb2-28ffacae5120 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:03.945098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:03.945098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:03.945098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:03.945098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:03.945098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:03.945098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:03.945098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:03.945098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:03.945098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:03.945098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:03.945098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:03.945098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:03.945098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:03.945098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:03.946024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:03.946024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:03.946024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:03.946024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:03.946024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:03.946024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:03.946024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:03.946024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:03.946024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:03.946024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:03.946024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:03.946024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:03.946024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:03.946024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:03.946024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:03.946024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:03.946024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:03.947145 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:03.947145 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:03.947145 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:03.947145 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:03.947145 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 316/644] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:03.947432 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebeadf6-ee8b-4c20-adb2-28ffacae5120 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:03.980489 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:04.047588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82608c18-cc06-4c6d-b6b0-1a51c8499d8a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:04.056841 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebeadf6-ee8b-4c20-adb2-28ffacae5120 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:04.059687 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:04.090662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60195ce-3b43-40a0-937b-c6525515b3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:04.091291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60195ce-3b43-40a0-937b-c6525515b3de None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:04.091538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60195ce-3b43-40a0-937b-c6525515b3de None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:04.091816 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60195ce-3b43-40a0-937b-c6525515b3de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:04.094971 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a212d7c0-05cc-4d76-b684-a0b4f1bc9b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:04.097046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212d7c0-05cc-4d76-b684-a0b4f1bc9b66 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:04.097361 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212d7c0-05cc-4d76-b684-a0b4f1bc9b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:04.097608 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212d7c0-05cc-4d76-b684-a0b4f1bc9b66 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:04.098385 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a212d7c0-05cc-4d76-b684-a0b4f1bc9b66 None admin] RBAC: Adding query filter params (name=service) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:04.098697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a212d7c0-05cc-4d76-b684-a0b4f1bc9b66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:04.100089 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a212d7c0-05cc-4d76-b684-a0b4f1bc9b66 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:04.110558 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 317/645] 10.222.0.76 () {56 vars in 1103 bytes} [Thu May 26 17:51:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:04.111461 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a212d7c0-05cc-4d76-b684-a0b4f1bc9b66 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:04.182078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60195ce-3b43-40a0-937b-c6525515b3de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:04.183689 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60195ce-3b43-40a0-937b-c6525515b3de None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:04.215891 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:04.227005 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a212d7c0-05cc-4d76-b684-a0b4f1bc9b66 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:04.265768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8d47e-063f-4767-bb83-e97f84bebd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:04.267043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8d47e-063f-4767-bb83-e97f84bebd66 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:04.267335 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8d47e-063f-4767-bb83-e97f84bebd66 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:04.267617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8d47e-063f-4767-bb83-e97f84bebd66 None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:04.269536 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8d47e-063f-4767-bb83-e97f84bebd66 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:04.269854 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8d47e-063f-4767-bb83-e97f84bebd66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:04.271984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8d47e-063f-4767-bb83-e97f84bebd66 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:04.298156 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 329/646] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:51:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:51:04.298156 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60195ce-3b43-40a0-937b-c6525515b3de None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:04.298853 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 318/647] 10.222.0.76 () {56 vars in 1135 bytes} [Thu May 26 17:51:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:04.299850 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8d47e-063f-4767-bb83-e97f84bebd66 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:04.309834 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:04.409973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8d47e-063f-4767-bb83-e97f84bebd66 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:04.410153 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60195ce-3b43-40a0-937b-c6525515b3de None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:04.447975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291355e-e633-40a2-9b42-ad15328e25e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:04.448321 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ad954-4665-4511-b306-90aa9b4e21fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:04.448775 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291355e-e633-40a2-9b42-ad15328e25e3 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:04.448983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ad954-4665-4511-b306-90aa9b4e21fc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:04.449085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291355e-e633-40a2-9b42-ad15328e25e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:04.449273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ad954-4665-4511-b306-90aa9b4e21fc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:04.449372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291355e-e633-40a2-9b42-ad15328e25e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:04.449553 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ad954-4665-4511-b306-90aa9b4e21fc None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:04.450299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ad954-4665-4511-b306-90aa9b4e21fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:04.450801 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ad954-4665-4511-b306-90aa9b4e21fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:04.452273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ad954-4665-4511-b306-90aa9b4e21fc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:04.461781 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8291355e-e633-40a2-9b42-ad15328e25e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:04.463948 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8291355e-e633-40a2-9b42-ad15328e25e3 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:04.465174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 330/648] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:04.466066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ad954-4665-4511-b306-90aa9b4e21fc None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:04.490736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-8291355e-e633-40a2-9b42-ad15328e25e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:04.490736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:04.490736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:04.490736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:04.490736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:04.490736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:04.490736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:04.490736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:04.490736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:04.490736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:04.490736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:04.490736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:04.490736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:04.490736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:04.490736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:04.491737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:04.491737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:04.491737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:04.491737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:04.491737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:04.491737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:04.491737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:04.491737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:04.491737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:04.491737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:04.491737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:04.491737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:04.491737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:04.491737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:04.491737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:04.491737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:04.491737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:04.492634 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:04.492634 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:04.492634 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:04.492634 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:04.493506 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 319/649] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:04.493960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291355e-e633-40a2-9b42-ad15328e25e3 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:04.496914 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:04.549630 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:04.583625 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ad954-4665-4511-b306-90aa9b4e21fc None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:04.614201 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291355e-e633-40a2-9b42-ad15328e25e3 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:04.619608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a65adb-5975-4b97-9b52-ba001a0b1048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:04.620408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a65adb-5975-4b97-9b52-ba001a0b1048 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:04.620693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a65adb-5975-4b97-9b52-ba001a0b1048 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:04.620966 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a65adb-5975-4b97-9b52-ba001a0b1048 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:04.632362 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a65adb-5975-4b97-9b52-ba001a0b1048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:04.633854 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a65adb-5975-4b97-9b52-ba001a0b1048 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:04.652476 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e72347-489e-46b6-a2cf-4d5cc58568c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:04.652701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-69a65adb-5975-4b97-9b52-ba001a0b1048 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:04.652701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:04.652701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:04.652701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:04.652701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:04.652701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:04.652701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:04.652701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:04.652701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:04.652701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:04.652701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:04.652701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:04.652701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:04.652701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:04.652701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:04.653619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:04.653619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:04.653619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:04.653619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:04.653619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:04.653619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:04.653619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:04.653619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:04.653619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:04.653619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:04.653619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:04.653619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:04.653619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:04.653619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:04.653619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:04.653619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:04.653619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:04.654611 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:04.654611 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:04.654611 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:04.654611 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:04.654611 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 331/650] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:04.654611 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a65adb-5975-4b97-9b52-ba001a0b1048 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:04.656254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e72347-489e-46b6-a2cf-4d5cc58568c2 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:04.656661 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e72347-489e-46b6-a2cf-4d5cc58568c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:04.657008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e72347-489e-46b6-a2cf-4d5cc58568c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:04.670643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e72347-489e-46b6-a2cf-4d5cc58568c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:04.672051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e72347-489e-46b6-a2cf-4d5cc58568c2 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:04.691136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-03e72347-489e-46b6-a2cf-4d5cc58568c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:04.691136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:04.691136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:04.691136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:04.691136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:04.691136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:04.691136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:04.691136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:04.691136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:04.691136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:04.691136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:04.691136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:04.691136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:04.691136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:04.691136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:04.692562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:04.692562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:04.692562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:04.692562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:04.692562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:04.692562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:04.692562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:04.692562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:04.692562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:04.692562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:04.692562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:04.692562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:04.692562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:04.692562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:04.692562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:04.692562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:04.692562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:04.693524 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:04.693524 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:04.693524 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:04.693524 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:04.693524 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 320/651] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:04.693524 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e72347-489e-46b6-a2cf-4d5cc58568c2 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:04.757267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:04.782199 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a65adb-5975-4b97-9b52-ba001a0b1048 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:04.809174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:04.811771 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e72347-489e-46b6-a2cf-4d5cc58568c2 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:04.824023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48444247-84bd-4823-b6c7-927515f02c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:04.824857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48444247-84bd-4823-b6c7-927515f02c4f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:04.825155 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48444247-84bd-4823-b6c7-927515f02c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:04.825432 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48444247-84bd-4823-b6c7-927515f02c4f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:04.833661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48444247-84bd-4823-b6c7-927515f02c4f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:04.835230 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48444247-84bd-4823-b6c7-927515f02c4f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:04.849734 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54655b1f-0a16-40d5-88d6-dc51021f1539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:04.850431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54655b1f-0a16-40d5-88d6-dc51021f1539 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:04.850538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-48444247-84bd-4823-b6c7-927515f02c4f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 17:51:04.850538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:04.850538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:04.850538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:04.850538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:04.850538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:04.850538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:04.850538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:04.850538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:04.850538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:04.850538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:04.850538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:04.850538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:51:04.850538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:51:04.850538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:51:04.850538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:51:04.851484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:04.851484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:04.851484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:51:04.851484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:04.851484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:51:04.851484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:04.851484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:04.851484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:04.851484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:04.851484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:04.851484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:04.851484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:04.851484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:04.851484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:04.851484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:04.851484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:04.851484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:04.851484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:04.852419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:04.852419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:04.852419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:51:04.852419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:51:04.852419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:51:04.852419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:51:04.852419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:51:04.852419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:51:04.852419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 17:51:04.852419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:04.852961 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54655b1f-0a16-40d5-88d6-dc51021f1539 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:04.852961 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54655b1f-0a16-40d5-88d6-dc51021f1539 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:04.852961 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54655b1f-0a16-40d5-88d6-dc51021f1539 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:04.853154 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 332/652] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:51:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:04.853154 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48444247-84bd-4823-b6c7-927515f02c4f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:04.856443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54655b1f-0a16-40d5-88d6-dc51021f1539 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:04.857868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54655b1f-0a16-40d5-88d6-dc51021f1539 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:04.874829 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 321/653] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:04.874829 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54655b1f-0a16-40d5-88d6-dc51021f1539 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:04.974474 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48444247-84bd-4823-b6c7-927515f02c4f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:04.983416 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54655b1f-0a16-40d5-88d6-dc51021f1539 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:04.998003 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:05.012376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc978574-c8df-494b-8abe-724fb49eecc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:05.013048 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc978574-c8df-494b-8abe-724fb49eecc9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:05.013342 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc978574-c8df-494b-8abe-724fb49eecc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:05.013626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc978574-c8df-494b-8abe-724fb49eecc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:05.014377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc978574-c8df-494b-8abe-724fb49eecc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:05.014688 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc978574-c8df-494b-8abe-724fb49eecc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:05.016060 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc978574-c8df-494b-8abe-724fb49eecc9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:05.022492 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82cf8a-21bc-4394-8ffa-3e31e67a7275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:05.023193 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82cf8a-21bc-4394-8ffa-3e31e67a7275 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:05.023503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82cf8a-21bc-4394-8ffa-3e31e67a7275 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:05.023825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82cf8a-21bc-4394-8ffa-3e31e67a7275 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:05.024625 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd82cf8a-21bc-4394-8ffa-3e31e67a7275 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:05.024978 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd82cf8a-21bc-4394-8ffa-3e31e67a7275 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:05.026428 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd82cf8a-21bc-4394-8ffa-3e31e67a7275 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:05.034336 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 333/654] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:05.035248 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc978574-c8df-494b-8abe-724fb49eecc9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:05.041173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 322/655] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:05.041865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82cf8a-21bc-4394-8ffa-3e31e67a7275 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:05.091837 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:05.161502 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc978574-c8df-494b-8abe-724fb49eecc9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:05.165800 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82cf8a-21bc-4394-8ffa-3e31e67a7275 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:05.199560 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d276ddc-eba3-4641-9e1c-3d5b6913ee5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:05.200231 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d276ddc-eba3-4641-9e1c-3d5b6913ee5c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:05.200574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d276ddc-eba3-4641-9e1c-3d5b6913ee5c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:05.201454 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d276ddc-eba3-4641-9e1c-3d5b6913ee5c None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:05.201709 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d276ddc-eba3-4641-9e1c-3d5b6913ee5c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:05.202015 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d276ddc-eba3-4641-9e1c-3d5b6913ee5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:05.203516 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d276ddc-eba3-4641-9e1c-3d5b6913ee5c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:05.204332 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadf203-fce5-4c7a-bcb6-6f62543dd903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:05.205285 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadf203-fce5-4c7a-bcb6-6f62543dd903 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:05.206305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadf203-fce5-4c7a-bcb6-6f62543dd903 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:05.206730 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadf203-fce5-4c7a-bcb6-6f62543dd903 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:05.214782 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadf203-fce5-4c7a-bcb6-6f62543dd903 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:05.216376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadf203-fce5-4c7a-bcb6-6f62543dd903 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:05.228306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 334/656] 10.222.0.76 () {56 vars in 1137 bytes} [Thu May 26 17:51:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:05.228306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d276ddc-eba3-4641-9e1c-3d5b6913ee5c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:05.232156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-aaadf203-fce5-4c7a-bcb6-6f62543dd903 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 17:51:05.232156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:05.232156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:05.232156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:05.232156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:05.232156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:05.232156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:05.232156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:05.232156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:05.232156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:05.232156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:05.232156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:05.232156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:51:05.232156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:51:05.232156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:51:05.232156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:51:05.233125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:05.233125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:05.233125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:51:05.233125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:05.233125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:51:05.233125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:05.233125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:05.233125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:05.233125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:05.233125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:05.233125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:05.233125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:05.233125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:05.233125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:05.233125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:05.233125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:05.233125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:05.233125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:05.234074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:05.234074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:05.234074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:51:05.234074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:51:05.234074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:51:05.234074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:51:05.234074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:51:05.234074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:51:05.234074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 17:51:05.234074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:05.234074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 323/657] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:51:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:05.234731 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadf203-fce5-4c7a-bcb6-6f62543dd903 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:05.274971 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:05.338304 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:05.341225 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d276ddc-eba3-4641-9e1c-3d5b6913ee5c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:05.345552 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadf203-fce5-4c7a-bcb6-6f62543dd903 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:05.377140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94153bb2-1d7e-4ad8-b035-14024e492cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:05.377961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94153bb2-1d7e-4ad8-b035-14024e492cab None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:05.378281 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94153bb2-1d7e-4ad8-b035-14024e492cab None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:05.378554 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94153bb2-1d7e-4ad8-b035-14024e492cab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:05.383618 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb97c1b-7419-4fcd-84a3-c5b758873c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:05.384419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb97c1b-7419-4fcd-84a3-c5b758873c6d None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:05.384702 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb97c1b-7419-4fcd-84a3-c5b758873c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:05.384976 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb97c1b-7419-4fcd-84a3-c5b758873c6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:05.385952 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94153bb2-1d7e-4ad8-b035-14024e492cab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:05.387709 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94153bb2-1d7e-4ad8-b035-14024e492cab None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:05.392511 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb97c1b-7419-4fcd-84a3-c5b758873c6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:05.394579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb97c1b-7419-4fcd-84a3-c5b758873c6d None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:05.403180 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-94153bb2-1d7e-4ad8-b035-14024e492cab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:05.403180 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:05.403180 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:05.403180 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:05.403180 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:05.403180 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:05.403180 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:05.403180 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:05.403180 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:05.403180 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:05.403180 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:05.403180 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:05.403180 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:51:05.403180 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:51:05.403180 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:51:05.404253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:51:05.404253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:05.404253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:05.404253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:05.404253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:05.404253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:05.404253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:05.404253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:05.404253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:05.404253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:05.404253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:05.404253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:05.404253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:05.404253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:05.404253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:05.404253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:05.404253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:05.411391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:51:05.411391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:51:05.411391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:51:05.411391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:51:05.411391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:51:05.411391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:51:05.411391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:05.411391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:05.411391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 335/658] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:05.411391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94153bb2-1d7e-4ad8-b035-14024e492cab None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:05.411960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-1cb97c1b-7419-4fcd-84a3-c5b758873c6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:05.411960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:05.411960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:05.411960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:05.411960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:05.411960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:05.411960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:05.411960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:05.411960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:05.411960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:05.411960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:05.411960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:05.411960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:51:05.411960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:51:05.411960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:51:05.412834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:51:05.412834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:05.412834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:05.412834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:05.412834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:05.412834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:05.412834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:05.412834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:05.412834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:05.412834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:05.412834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:05.412834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:05.412834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:05.412834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:05.412834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:05.412834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:05.412834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:05.413758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:51:05.413758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:51:05.413758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:51:05.413758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:51:05.413758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:51:05.413758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:51:05.413758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:05.413758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:05.413758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 324/659] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:05.413758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb97c1b-7419-4fcd-84a3-c5b758873c6d None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:05.508610 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94153bb2-1d7e-4ad8-b035-14024e492cab None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:05.511523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:05.520571 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb97c1b-7419-4fcd-84a3-c5b758873c6d None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:05.546588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd03330-b9ef-40c4-ba5a-5ffd08530df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:05.547248 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd03330-b9ef-40c4-ba5a-5ffd08530df6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:05.547536 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd03330-b9ef-40c4-ba5a-5ffd08530df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:05.547817 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd03330-b9ef-40c4-ba5a-5ffd08530df6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:05.555990 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63db84-4a82-4bb6-977a-0421dfc3e1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:05.557047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63db84-4a82-4bb6-977a-0421dfc3e1cd None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:05.557047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63db84-4a82-4bb6-977a-0421dfc3e1cd None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:05.557234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63db84-4a82-4bb6-977a-0421dfc3e1cd None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:05.558022 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63db84-4a82-4bb6-977a-0421dfc3e1cd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:05.558366 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63db84-4a82-4bb6-977a-0421dfc3e1cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:05.559820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63db84-4a82-4bb6-977a-0421dfc3e1cd None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:05.577008 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:05.577936 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 325/660] 10.222.0.76 () {56 vars in 1139 bytes} [Thu May 26 17:51:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:05.579296 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c2c62-85d2-4eb3-9093-a1ff207f1b91 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:05.579590 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c2c62-85d2-4eb3-9093-a1ff207f1b91 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:05.579873 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c2c62-85d2-4eb3-9093-a1ff207f1b91 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:05.581732 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 326/661] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:05.582377 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c2c62-85d2-4eb3-9093-a1ff207f1b91 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:05.622574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd03330-b9ef-40c4-ba5a-5ffd08530df6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:05.624029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd03330-b9ef-40c4-ba5a-5ffd08530df6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:05.694118 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c2c62-85d2-4eb3-9093-a1ff207f1b91 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:05.731965 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 336/662] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:51:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:51:05.732212 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf61ca-95b9-4371-b0a2-9386329b086e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:05.732722 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf61ca-95b9-4371-b0a2-9386329b086e None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:05.732831 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd03330-b9ef-40c4-ba5a-5ffd08530df6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:05.733017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf61ca-95b9-4371-b0a2-9386329b086e None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:05.733308 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf61ca-95b9-4371-b0a2-9386329b086e None admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:05.734055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf61ca-95b9-4371-b0a2-9386329b086e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:05.734399 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf61ca-95b9-4371-b0a2-9386329b086e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:05.735831 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf61ca-95b9-4371-b0a2-9386329b086e None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:05.749198 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 327/663] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:51:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:05.750507 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302c2b0-c67f-4f6a-b501-1ed75ee27fd2 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:05.750798 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302c2b0-c67f-4f6a-b501-1ed75ee27fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:05.751065 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302c2b0-c67f-4f6a-b501-1ed75ee27fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:05.770448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-8302c2b0-c67f-4f6a-b501-1ed75ee27fd2 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:05.771598 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-8302c2b0-c67f-4f6a-b501-1ed75ee27fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:05.780831 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:05.798430 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:05.851875 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd03330-b9ef-40c4-ba5a-5ffd08530df6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:05.859440 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 328/664] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:51:05.860004 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302c2b0-c67f-4f6a-b501-1ed75ee27fd2 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:05.888243 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1910925-aa1e-49c0-b42b-0fa3f1cdb679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:05.888920 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1910925-aa1e-49c0-b42b-0fa3f1cdb679 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:05.889204 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1910925-aa1e-49c0-b42b-0fa3f1cdb679 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:05.889493 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1910925-aa1e-49c0-b42b-0fa3f1cdb679 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:05.890310 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1910925-aa1e-49c0-b42b-0fa3f1cdb679 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:05.890628 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1910925-aa1e-49c0-b42b-0fa3f1cdb679 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:05.892124 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1910925-aa1e-49c0-b42b-0fa3f1cdb679 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:05.905015 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 337/665] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:51:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:05.905689 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1910925-aa1e-49c0-b42b-0fa3f1cdb679 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:05.981711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302c2b0-c67f-4f6a-b501-1ed75ee27fd2 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:06.016970 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3ff4c-41ef-40c4-9048-c89cfdb3028d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:06.017621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3ff4c-41ef-40c4-9048-c89cfdb3028d None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:06.017903 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3ff4c-41ef-40c4-9048-c89cfdb3028d None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:06.018829 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3ff4c-41ef-40c4-9048-c89cfdb3028d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:06.018979 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1910925-aa1e-49c0-b42b-0fa3f1cdb679 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:06.031177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:06.042073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:06.065602 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d873124-5df2-43db-954d-9bef1b451a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:06.066804 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d873124-5df2-43db-954d-9bef1b451a47 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:06.067322 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d873124-5df2-43db-954d-9bef1b451a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:06.067829 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d873124-5df2-43db-954d-9bef1b451a47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:06.086622 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d873124-5df2-43db-954d-9bef1b451a47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:06.089337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d873124-5df2-43db-954d-9bef1b451a47 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:06.090343 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de3ff4c-41ef-40c4-9048-c89cfdb3028d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:06.091783 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de3ff4c-41ef-40c4-9048-c89cfdb3028d None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:06.124109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-2d873124-5df2-43db-954d-9bef1b451a47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:06.124109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:06.124109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:06.124109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:06.124109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:06.124109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:06.124109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:06.124109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:06.124109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:06.124109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:06.124109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:06.124109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:06.124109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:06.124109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:06.124109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:06.125569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:06.125569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:06.125569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:06.125569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:06.125569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:06.125569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:06.125569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:06.125569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:06.125569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:06.125569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:06.125569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:06.125569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:06.125569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:06.125569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:06.125569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:06.125569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:06.125569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:06.126705 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:06.126705 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:06.126705 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:06.126705 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:06.126705 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 338/666] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:06.128607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d873124-5df2-43db-954d-9bef1b451a47 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:06.206810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 329/667] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:51:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:51:06.207709 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3ff4c-41ef-40c4-9048-c89cfdb3028d None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:06.244206 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d873124-5df2-43db-954d-9bef1b451a47 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:06.255838 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:06.282488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cb69c-2472-41ed-ab4b-5d996fa30c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:06.283054 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cb69c-2472-41ed-ab4b-5d996fa30c2f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:06.283342 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cb69c-2472-41ed-ab4b-5d996fa30c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:06.283647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cb69c-2472-41ed-ab4b-5d996fa30c2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:06.284649 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220cb69c-2472-41ed-ab4b-5d996fa30c2f None admin] RBAC: Adding query filter params (role.id=e776303775a2423aa0c62c161ecb5a7e, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=1899c317083d40f5b490cd9eb63fe521) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:06.284967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220cb69c-2472-41ed-ab4b-5d996fa30c2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:06.286740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220cb69c-2472-41ed-ab4b-5d996fa30c2f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:06.294063 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 339/668] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:51:06 2022] GET /identity/v3/role_assignments?user.id=1899c317083d40f5b490cd9eb63fe521&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=e776303775a2423aa0c62c161ecb5a7e => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:06.295378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cb69c-2472-41ed-ab4b-5d996fa30c2f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:06.322634 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3ff4c-41ef-40c4-9048-c89cfdb3028d None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:06.348211 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:06.369579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84846f7-cafa-4897-be13-0263a5b88d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:06.370439 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84846f7-cafa-4897-be13-0263a5b88d89 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:06.370730 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84846f7-cafa-4897-be13-0263a5b88d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:06.371011 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84846f7-cafa-4897-be13-0263a5b88d89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:06.383901 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84846f7-cafa-4897-be13-0263a5b88d89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:06.385574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84846f7-cafa-4897-be13-0263a5b88d89 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:06.405591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-b84846f7-cafa-4897-be13-0263a5b88d89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:06.405591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:06.405591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:06.405591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:06.405591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:06.405591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:06.405591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:06.405591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:06.405591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:06.405591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:06.405591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:06.405591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:06.405591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:06.405591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:06.405591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:06.406630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:06.406630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:06.406630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:06.406630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:06.406630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:06.406630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:06.406630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:06.406630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:06.406630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:06.406630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:06.406630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:06.406630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:06.406630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:06.406630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:06.406630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:06.406630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:06.406630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:06.407537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:06.407537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:06.407537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:06.407537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:06.407537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 330/669] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:06.407811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84846f7-cafa-4897-be13-0263a5b88d89 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:06.431766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cb69c-2472-41ed-ab4b-5d996fa30c2f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:06.477709 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a94e6b-5aeb-4a93-87f1-6d82a8f8f777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:06.478637 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a94e6b-5aeb-4a93-87f1-6d82a8f8f777 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:06.478913 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a94e6b-5aeb-4a93-87f1-6d82a8f8f777 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:06.479399 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a94e6b-5aeb-4a93-87f1-6d82a8f8f777 None admin] PATH_INFO: `/v3/projects/ca1d09f491d944f99425a641561b03c3/users/9ca774e864ca44cfb187e38bc8a8f6ea/roles/e776303775a2423aa0c62c161ecb5a7e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:06.497295 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a94e6b-5aeb-4a93-87f1-6d82a8f8f777 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1d09f491d944f99425a641561b03c3, user_id=9ca774e864ca44cfb187e38bc8a8f6ea, role_id=e776303775a2423aa0c62c161ecb5a7e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:06.499264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a94e6b-5aeb-4a93-87f1-6d82a8f8f777 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:06.516411 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 340/670] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:51:06 2022] PUT /identity/v3/projects/ca1d09f491d944f99425a641561b03c3/users/9ca774e864ca44cfb187e38bc8a8f6ea/roles/e776303775a2423aa0c62c161ecb5a7e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:51:06.518118 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a94e6b-5aeb-4a93-87f1-6d82a8f8f777 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:06.527395 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:06.539545 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84846f7-cafa-4897-be13-0263a5b88d89 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:06.596767 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e64f2-7fee-46d1-88f7-de762cf1468a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:06.598045 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e64f2-7fee-46d1-88f7-de762cf1468a None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:06.599064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e64f2-7fee-46d1-88f7-de762cf1468a None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:06.599728 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e64f2-7fee-46d1-88f7-de762cf1468a None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:06.600818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e64f2-7fee-46d1-88f7-de762cf1468a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:06.601517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e64f2-7fee-46d1-88f7-de762cf1468a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:06.603127 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e64f2-7fee-46d1-88f7-de762cf1468a None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:06.620890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 331/671] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:06.622006 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e64f2-7fee-46d1-88f7-de762cf1468a None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:06.649058 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:06.672249 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a94e6b-5aeb-4a93-87f1-6d82a8f8f777 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:06.715329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92f523-3043-4156-8667-b66e2321251e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:06.716141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92f523-3043-4156-8667-b66e2321251e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:06.716479 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92f523-3043-4156-8667-b66e2321251e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:06.716763 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92f523-3043-4156-8667-b66e2321251e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:06.732841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92f523-3043-4156-8667-b66e2321251e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:06.734701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92f523-3043-4156-8667-b66e2321251e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:06.755466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-cd92f523-3043-4156-8667-b66e2321251e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:06.755466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:06.755466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:06.755466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:06.755466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:06.755466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:06.755466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:06.755466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:06.755466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:06.755466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:06.755466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:06.755466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:06.755466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:06.755466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:06.755466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:06.756418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:06.756418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:06.756418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:06.756418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:06.756418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:06.756418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:06.756418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:06.756418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:06.756418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:06.756418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:06.756418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:06.756418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:06.756418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:06.756418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:06.756418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:06.756418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:06.756418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:06.758351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:06.758351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:06.758351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:06.758351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:06.758351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 341/672] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:06.758351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92f523-3043-4156-8667-b66e2321251e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:06.774489 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e64f2-7fee-46d1-88f7-de762cf1468a None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:06.818942 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15716da-8572-4030-816a-5ff3e56ee77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:06.819958 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15716da-8572-4030-816a-5ff3e56ee77a None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:06.820440 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15716da-8572-4030-816a-5ff3e56ee77a None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:06.820912 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15716da-8572-4030-816a-5ff3e56ee77a None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:06.821824 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15716da-8572-4030-816a-5ff3e56ee77a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:06.822358 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15716da-8572-4030-816a-5ff3e56ee77a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:06.824059 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15716da-8572-4030-816a-5ff3e56ee77a None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:06.837670 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 332/673] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:06.838935 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15716da-8572-4030-816a-5ff3e56ee77a None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:06.856416 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:06.879105 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92f523-3043-4156-8667-b66e2321251e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:06.917897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db399281-327d-4ef0-831c-ca55938862ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:06.919401 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db399281-327d-4ef0-831c-ca55938862ed None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:06.919401 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db399281-327d-4ef0-831c-ca55938862ed None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:06.919401 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db399281-327d-4ef0-831c-ca55938862ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:06.927193 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db399281-327d-4ef0-831c-ca55938862ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:06.928851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db399281-327d-4ef0-831c-ca55938862ed None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:06.930780 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:06.945066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-db399281-327d-4ef0-831c-ca55938862ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:06.945066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:06.945066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:06.945066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:06.945066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:06.945066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:06.945066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:06.945066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:06.945066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:06.945066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:06.945066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:06.945066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:06.945066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:51:06.945066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:51:06.945066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:51:06.946004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:51:06.946004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:06.946004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:06.946004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:06.946004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:06.946004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:06.946004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:06.946004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:06.946004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:06.946004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:06.946004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:06.946004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:06.946004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:06.946004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:06.946004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:06.946004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:06.946004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:06.946942 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:51:06.946942 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:51:06.946942 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:51:06.946942 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:51:06.946942 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:51:06.946942 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:51:06.946942 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:06.946942 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:06.950640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 342/674] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:06.951720 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db399281-327d-4ef0-831c-ca55938862ed None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:06.955062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15716da-8572-4030-816a-5ff3e56ee77a None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:06.993382 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6c68c-746c-4dfe-be64-4e6b1b632bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:06.994377 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6c68c-746c-4dfe-be64-4e6b1b632bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:06.994800 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6c68c-746c-4dfe-be64-4e6b1b632bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:06.995234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6c68c-746c-4dfe-be64-4e6b1b632bc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:06.996107 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6c68c-746c-4dfe-be64-4e6b1b632bc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:06.996556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6c68c-746c-4dfe-be64-4e6b1b632bc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:06.998140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6c68c-746c-4dfe-be64-4e6b1b632bc0 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:07.015436 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 333/675] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:07.016568 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6c68c-746c-4dfe-be64-4e6b1b632bc0 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:07.061475 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db399281-327d-4ef0-831c-ca55938862ed None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:07.101666 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:07.148471 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6c68c-746c-4dfe-be64-4e6b1b632bc0 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:07.189624 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a742d-a0ff-4115-8a39-4d510b17948f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:07.191195 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a742d-a0ff-4115-8a39-4d510b17948f None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:07.191809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a742d-a0ff-4115-8a39-4d510b17948f None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:07.192266 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a742d-a0ff-4115-8a39-4d510b17948f None admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:07.193227 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a742d-a0ff-4115-8a39-4d510b17948f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:07.193692 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a742d-a0ff-4115-8a39-4d510b17948f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:07.195388 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a742d-a0ff-4115-8a39-4d510b17948f None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:07.209087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 334/676] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:51:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:07.210162 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a742d-a0ff-4115-8a39-4d510b17948f None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:07.231687 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fbe42b-2ba6-4faf-a57d-1261b79915d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:07.232364 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbe42b-2ba6-4faf-a57d-1261b79915d2 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:07.232649 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbe42b-2ba6-4faf-a57d-1261b79915d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:07.232924 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbe42b-2ba6-4faf-a57d-1261b79915d2 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:07.233635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fbe42b-2ba6-4faf-a57d-1261b79915d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:07.235245 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fbe42b-2ba6-4faf-a57d-1261b79915d2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:07.266270 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 343/677] 10.222.0.76 () {60 vars in 1132 bytes} [Thu May 26 17:51:06 2022] POST /identity/v3/users => generated 254 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:51:07.266270 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fbe42b-2ba6-4faf-a57d-1261b79915d2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:07.340013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:07.397376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a742d-a0ff-4115-8a39-4d510b17948f None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:07.402224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fbe42b-2ba6-4faf-a57d-1261b79915d2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:07.444564 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba7b8b-5e19-4ec4-aee2-6760a1553e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:07.445229 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba7b8b-5e19-4ec4-aee2-6760a1553e0f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:07.445513 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba7b8b-5e19-4ec4-aee2-6760a1553e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:07.445789 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba7b8b-5e19-4ec4-aee2-6760a1553e0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:07.446653 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba7b8b-5e19-4ec4-aee2-6760a1553e0f None admin] RBAC: Adding query filter params (role.id=e776303775a2423aa0c62c161ecb5a7e, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=76dbf8f3f1b44baca2d2aa90fe606a30) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:07.446965 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba7b8b-5e19-4ec4-aee2-6760a1553e0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:07.449018 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba7b8b-5e19-4ec4-aee2-6760a1553e0f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:07.457490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 344/678] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:51:07 2022] GET /identity/v3/role_assignments?user.id=76dbf8f3f1b44baca2d2aa90fe606a30&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=e776303775a2423aa0c62c161ecb5a7e => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:07.489746 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1011338-8bf7-4903-ad6e-9a26fdf7d898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:07.490966 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1011338-8bf7-4903-ad6e-9a26fdf7d898 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:07.491405 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1011338-8bf7-4903-ad6e-9a26fdf7d898 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:07.491832 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1011338-8bf7-4903-ad6e-9a26fdf7d898 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:07.500503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:07.511617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1011338-8bf7-4903-ad6e-9a26fdf7d898 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:07.513149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1011338-8bf7-4903-ad6e-9a26fdf7d898 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:07.532427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-a1011338-8bf7-4903-ad6e-9a26fdf7d898 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:07.532427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:07.532427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:07.532427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:07.532427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:07.532427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:07.532427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:07.532427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:07.532427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:07.532427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:07.532427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:07.532427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:07.532427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:51:07.532427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:51:07.532427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:51:07.533567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:51:07.533567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:07.533567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:07.533567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:07.533567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:07.533567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:07.533567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:07.533567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:07.533567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:07.533567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:07.533567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:07.533567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:07.533567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:07.533567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:07.533567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:07.533567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:07.533567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:07.535156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:51:07.535156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:51:07.535156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:51:07.535156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:51:07.535156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:51:07.535156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:51:07.535156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:07.535156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:07.535156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 335/679] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:07.540872 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba7b8b-5e19-4ec4-aee2-6760a1553e0f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:07.691966 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba7b8b-5e19-4ec4-aee2-6760a1553e0f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:07.711640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:07.734863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e1ae6-2255-4cc0-b4de-c981a915b96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:07.734863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e1ae6-2255-4cc0-b4de-c981a915b96c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:07.734863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e1ae6-2255-4cc0-b4de-c981a915b96c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:07.735229 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e1ae6-2255-4cc0-b4de-c981a915b96c None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:07.735851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e1ae6-2255-4cc0-b4de-c981a915b96c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:07.736163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e1ae6-2255-4cc0-b4de-c981a915b96c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:07.738367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e1ae6-2255-4cc0-b4de-c981a915b96c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:07.753535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 345/680] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:51:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:07.759512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1011338-8bf7-4903-ad6e-9a26fdf7d898 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:07.873230 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1011338-8bf7-4903-ad6e-9a26fdf7d898 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:07.910823 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd4b50-7462-4960-9ece-72765bcb213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:07.911749 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd4b50-7462-4960-9ece-72765bcb213b None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:07.912034 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd4b50-7462-4960-9ece-72765bcb213b None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:07.912313 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd4b50-7462-4960-9ece-72765bcb213b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:07.913154 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd4b50-7462-4960-9ece-72765bcb213b None admin] RBAC: Adding query filter params (role.id=e776303775a2423aa0c62c161ecb5a7e, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=8df66990a3734209a350c8b6768d259f) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:07.913464 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd4b50-7462-4960-9ece-72765bcb213b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:07.914943 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd4b50-7462-4960-9ece-72765bcb213b None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:07.928549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 336/681] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:51:07 2022] GET /identity/v3/role_assignments?user.id=8df66990a3734209a350c8b6768d259f&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=e776303775a2423aa0c62c161ecb5a7e => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:08.910463 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f26f4f-5659-4e81-8daa-c9744740b331 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:08.911234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f26f4f-5659-4e81-8daa-c9744740b331 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:08.911321 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f26f4f-5659-4e81-8daa-c9744740b331 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:08.912980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 346/682] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:08.918718 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08500a8d-a8c5-4072-9a4d-cb3aa1d9cdef None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:08.919214 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08500a8d-a8c5-4072-9a4d-cb3aa1d9cdef None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:08.919415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08500a8d-a8c5-4072-9a4d-cb3aa1d9cdef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:08.947302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-08500a8d-a8c5-4072-9a4d-cb3aa1d9cdef None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:08.948535 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-08500a8d-a8c5-4072-9a4d-cb3aa1d9cdef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:09.023301 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:09.076768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 337/683] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:51:09.126228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f26f4f-5659-4e81-8daa-c9744740b331 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:09.338818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c00aa-d430-4bf4-9b9d-efe5608e06cf None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:09.339254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c00aa-d430-4bf4-9b9d-efe5608e06cf None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:09.339361 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c00aa-d430-4bf4-9b9d-efe5608e06cf None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:09.340843 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 338/684] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:09.348439 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4673c6-4974-4750-94a3-3ec2aa635f48 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:09.348720 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4673c6-4974-4750-94a3-3ec2aa635f48 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:09.349164 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4673c6-4974-4750-94a3-3ec2aa635f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:09.391429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-6f4673c6-4974-4750-94a3-3ec2aa635f48 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:09.393462 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-6f4673c6-4974-4750-94a3-3ec2aa635f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:09.409916 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f26f4f-5659-4e81-8daa-c9744740b331 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:09.530932 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d52316-367b-46be-a7df-6ab29def0854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:09.532187 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d52316-367b-46be-a7df-6ab29def0854 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:09.532616 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d52316-367b-46be-a7df-6ab29def0854 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:09.533034 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d52316-367b-46be-a7df-6ab29def0854 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:09.575151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d52316-367b-46be-a7df-6ab29def0854 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:51:09.597634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-49d52316-367b-46be-a7df-6ab29def0854 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:51:09.597634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:09.597634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:09.597634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:09.597634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:09.597634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:09.597634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:09.597634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:09.597634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:09.597634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:09.597634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:09.597634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:09.597634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:51:09.597634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:51:09.597634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:51:09.597634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:51:09.598733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:51:09.598733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:51:09.598733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:51:09.598733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:51:09.598733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:09.598733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:09.598733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:09.598733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:09.598733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:09.598733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:09.598733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:09.598733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:09.598733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:09.598733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:09.598733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:09.598733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:09.599621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:09.599621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:09.599621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:09.599621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:09.599621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:51:09.599621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:51:09.599621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:51:09.599621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:51:09.599621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:51:09.599621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:51:09.599621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:51:09.599621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:09.610721 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 347/685] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:51:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 487 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:09.613552 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d52316-367b-46be-a7df-6ab29def0854 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:09.621045 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 339/686] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:51:09.662333 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4673c6-4974-4750-94a3-3ec2aa635f48 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:09.677898 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:09.796207 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d52316-367b-46be-a7df-6ab29def0854 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:09.889693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e1f08-71ba-4fe2-b57c-2e368f4a7f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:09.890454 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e1f08-71ba-4fe2-b57c-2e368f4a7f8f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:09.890741 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e1f08-71ba-4fe2-b57c-2e368f4a7f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:09.891258 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e1f08-71ba-4fe2-b57c-2e368f4a7f8f None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:09.892044 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e1f08-71ba-4fe2-b57c-2e368f4a7f8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:09.892353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e1f08-71ba-4fe2-b57c-2e368f4a7f8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:09.895021 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e1f08-71ba-4fe2-b57c-2e368f4a7f8f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:09.913068 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 348/687] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:51:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:09.923426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e1f08-71ba-4fe2-b57c-2e368f4a7f8f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:09.928649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4673c6-4974-4750-94a3-3ec2aa635f48 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:09.948431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:09.987946 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800a790-551e-4e88-ab57-baeebe17c69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:09.990475 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800a790-551e-4e88-ab57-baeebe17c69c None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:09.990925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800a790-551e-4e88-ab57-baeebe17c69c None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:09.991380 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800a790-551e-4e88-ab57-baeebe17c69c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:10.011091 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0800a790-551e-4e88-ab57-baeebe17c69c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:51:10.030221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-0800a790-551e-4e88-ab57-baeebe17c69c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:51:10.030221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:10.030221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:10.030221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:10.030221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:10.030221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:10.030221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:10.030221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:10.030221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:10.030221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:10.030221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:10.030221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:10.030221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:51:10.030221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:51:10.030221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:51:10.030221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:51:10.031773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:51:10.031773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:51:10.031773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:51:10.031773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:51:10.031773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:10.031773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:10.031773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:10.031773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:10.031773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:10.031773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:10.031773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:10.031773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:10.031773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:10.031773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:10.031773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:10.031773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:10.032930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:10.032930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:10.032930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:10.032930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:10.032930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:51:10.032930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:51:10.032930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:51:10.032930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:51:10.032930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:51:10.032930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:51:10.032930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:51:10.032930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:10.036746 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 340/688] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:51:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:51:10.045803 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800a790-551e-4e88-ab57-baeebe17c69c None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:10.150609 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:10.162844 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e1f08-71ba-4fe2-b57c-2e368f4a7f8f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:10.244213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4df91-aabb-4edc-b875-bf799abcda13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:10.245450 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4df91-aabb-4edc-b875-bf799abcda13 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:10.245979 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4df91-aabb-4edc-b875-bf799abcda13 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:10.246491 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4df91-aabb-4edc-b875-bf799abcda13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:10.259863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4df91-aabb-4edc-b875-bf799abcda13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:10.261799 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4df91-aabb-4edc-b875-bf799abcda13 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:10.264669 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800a790-551e-4e88-ab57-baeebe17c69c None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:10.282431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-e1a4df91-aabb-4edc-b875-bf799abcda13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:10.282431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:10.282431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:10.282431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:10.282431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:10.282431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:10.282431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:10.282431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:10.282431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:10.282431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:10.282431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:10.282431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:10.282431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:10.282431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:10.282431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:10.283383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:10.283383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:10.283383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:10.283383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:10.283383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:10.283383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:10.283383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:10.283383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:10.283383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:10.283383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:10.283383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:10.283383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:10.283383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:10.283383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:10.283383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:10.283383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:10.283383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:10.284975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:10.284975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:10.284975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:10.284975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:10.287549 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 349/689] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:10.293782 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4df91-aabb-4edc-b875-bf799abcda13 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:10.334979 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f598be-7a6e-4a79-bcdb-d0175aa3e6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:10.335688 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f598be-7a6e-4a79-bcdb-d0175aa3e6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:10.335983 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f598be-7a6e-4a79-bcdb-d0175aa3e6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:10.336274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f598be-7a6e-4a79-bcdb-d0175aa3e6f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:10.337049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f598be-7a6e-4a79-bcdb-d0175aa3e6f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:10.337365 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f598be-7a6e-4a79-bcdb-d0175aa3e6f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:10.338836 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f598be-7a6e-4a79-bcdb-d0175aa3e6f1 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:10.358334 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 341/690] 10.222.0.76 () {56 vars in 1103 bytes} [Thu May 26 17:51:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:10.367256 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f598be-7a6e-4a79-bcdb-d0175aa3e6f1 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:10.419502 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4df91-aabb-4edc-b875-bf799abcda13 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:10.460342 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a7dfd-1370-490f-9203-9932d94065f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:10.461489 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a7dfd-1370-490f-9203-9932d94065f9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:10.461987 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a7dfd-1370-490f-9203-9932d94065f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:10.462583 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a7dfd-1370-490f-9203-9932d94065f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:10.463370 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:10.465103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a7dfd-1370-490f-9203-9932d94065f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:10.466155 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a7dfd-1370-490f-9203-9932d94065f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:10.468592 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a7dfd-1370-490f-9203-9932d94065f9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:10.485945 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 350/691] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:10.488659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e36a3c-5510-4fa1-afee-fa1415833b4b None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:10.489251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e36a3c-5510-4fa1-afee-fa1415833b4b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:10.489794 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e36a3c-5510-4fa1-afee-fa1415833b4b None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:10.492041 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 351/692] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:10.493107 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e36a3c-5510-4fa1-afee-fa1415833b4b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:10.509823 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:10.554753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f598be-7a6e-4a79-bcdb-d0175aa3e6f1 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:10.609817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775b041-4100-4586-bf24-f21e34d72a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:10.610946 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775b041-4100-4586-bf24-f21e34d72a20 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:10.611234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775b041-4100-4586-bf24-f21e34d72a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:10.611549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775b041-4100-4586-bf24-f21e34d72a20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:10.613831 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e36a3c-5510-4fa1-afee-fa1415833b4b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:10.626061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a775b041-4100-4586-bf24-f21e34d72a20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:10.628350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a775b041-4100-4586-bf24-f21e34d72a20 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:10.650450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-a775b041-4100-4586-bf24-f21e34d72a20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:10.650450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:10.650450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:10.650450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:10.650450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:10.650450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:10.650450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:10.650450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:10.650450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:10.650450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:10.650450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:10.650450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:10.650450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:10.650450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:10.650450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:10.651626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:10.651626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:10.651626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:10.651626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:10.651626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:10.651626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:10.651626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:10.651626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:10.651626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:10.651626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:10.651626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:10.651626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:10.651626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:10.651626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:10.651626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:10.651626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:10.651626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:10.654306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:10.654306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:10.654306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:10.654306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:10.654306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 342/693] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:10.656389 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e04ca9-7077-4694-ac5b-04bdd56480fb None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:10.656782 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e04ca9-7077-4694-ac5b-04bdd56480fb None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:10.657110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e04ca9-7077-4694-ac5b-04bdd56480fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:10.671816 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d6ea7-670d-4d7e-85dc-5a22729773c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:10.672514 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d6ea7-670d-4d7e-85dc-5a22729773c6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:10.672799 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d6ea7-670d-4d7e-85dc-5a22729773c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:10.673098 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d6ea7-670d-4d7e-85dc-5a22729773c6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:10.682644 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d6ea7-670d-4d7e-85dc-5a22729773c6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:10.683423 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-39e04ca9-7077-4694-ac5b-04bdd56480fb None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:10.684102 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d6ea7-670d-4d7e-85dc-5a22729773c6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:10.685438 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-39e04ca9-7077-4694-ac5b-04bdd56480fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:10.700077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-705d6ea7-670d-4d7e-85dc-5a22729773c6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 17:51:10.700077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:10.700077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:10.700077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:10.700077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:10.700077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:10.700077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:10.700077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:10.700077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:10.700077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:10.700077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:10.700077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:10.700077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:51:10.700077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:51:10.700077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:51:10.700077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:51:10.701195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:10.701195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:10.701195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:51:10.701195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:10.701195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:51:10.701195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:10.701195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:10.701195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:10.701195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:10.701195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:10.701195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:10.701195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:10.701195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:10.701195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:10.701195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:10.701195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:10.701195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:10.701195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:10.702212 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:10.702212 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:10.702212 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:51:10.702212 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:51:10.702212 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:51:10.702212 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:51:10.702212 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:51:10.702212 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:51:10.702212 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 17:51:10.702212 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:10.702212 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 352/694] 10.222.0.76 () {56 vars in 1093 bytes} [Thu May 26 17:51:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:10.703445 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed66fd-3bf1-41c7-a209-324608a5729f None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:10.703445 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed66fd-3bf1-41c7-a209-324608a5729f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:10.703445 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed66fd-3bf1-41c7-a209-324608a5729f None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:10.708658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 353/695] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:10.708658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed66fd-3bf1-41c7-a209-324608a5729f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:10.787709 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:10.820634 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 343/696] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:51:10.822121 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e04ca9-7077-4694-ac5b-04bdd56480fb None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:10.831299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed66fd-3bf1-41c7-a209-324608a5729f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:10.865451 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:10.885320 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afb045-4ef5-4b47-93a3-355f55d394a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:10.886556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afb045-4ef5-4b47-93a3-355f55d394a8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:10.886847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afb045-4ef5-4b47-93a3-355f55d394a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:10.887167 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afb045-4ef5-4b47-93a3-355f55d394a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:10.887912 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0afb045-4ef5-4b47-93a3-355f55d394a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:10.888443 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0afb045-4ef5-4b47-93a3-355f55d394a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:10.889647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0afb045-4ef5-4b47-93a3-355f55d394a8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:10.901999 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 354/697] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:10.907681 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839c77c-0e8b-4e0f-af5d-206e148191a9 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:10.907681 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839c77c-0e8b-4e0f-af5d-206e148191a9 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:10.907681 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839c77c-0e8b-4e0f-af5d-206e148191a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:10.925290 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-7839c77c-0e8b-4e0f-af5d-206e148191a9 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:10.926538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7839c77c-0e8b-4e0f-af5d-206e148191a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:10.971660 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e04ca9-7077-4694-ac5b-04bdd56480fb None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:11.021745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 355/698] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:51:11.022926 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839c77c-0e8b-4e0f-af5d-206e148191a9 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:11.034546 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb026c1-9deb-460d-a33b-ad078c0789d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:11.036329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb026c1-9deb-460d-a33b-ad078c0789d2 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:11.037073 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb026c1-9deb-460d-a33b-ad078c0789d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:11.037792 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb026c1-9deb-460d-a33b-ad078c0789d2 None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:11.039554 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb026c1-9deb-460d-a33b-ad078c0789d2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:11.040377 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb026c1-9deb-460d-a33b-ad078c0789d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:11.043091 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb026c1-9deb-460d-a33b-ad078c0789d2 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:11.073778 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 344/699] 10.222.0.76 () {56 vars in 1133 bytes} [Thu May 26 17:51:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:11.075079 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb026c1-9deb-460d-a33b-ad078c0789d2 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:11.135729 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:11.188618 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839c77c-0e8b-4e0f-af5d-206e148191a9 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:11.194229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:11.216280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb026c1-9deb-460d-a33b-ad078c0789d2 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:11.227226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358e5dc7-78db-4e6b-adfa-29675d6a5e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:11.228025 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e5dc7-78db-4e6b-adfa-29675d6a5e49 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:11.228321 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e5dc7-78db-4e6b-adfa-29675d6a5e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:11.228590 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e5dc7-78db-4e6b-adfa-29675d6a5e49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:11.241435 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358e5dc7-78db-4e6b-adfa-29675d6a5e49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:51:11.249283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-358e5dc7-78db-4e6b-adfa-29675d6a5e49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:51:11.249283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:11.249283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:11.249283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:11.249283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:11.249283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:11.249283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:11.249283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:11.249283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:11.249283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:11.249283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:11.249283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:11.249283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:51:11.249283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:51:11.249283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:51:11.249283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:51:11.250492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:51:11.250492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:51:11.250492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:51:11.250492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:51:11.250492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:11.250492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:11.250492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:11.250492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:11.250492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:11.250492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:11.250492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:11.250492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:11.250492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:11.250492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:11.250492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:11.250492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:11.251460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:11.251460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:11.251460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:11.251460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:11.251460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:51:11.251460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:51:11.251460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:51:11.251460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:51:11.251460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:51:11.251460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:51:11.251460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:51:11.251460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:11.251460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 356/700] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:51:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:11.251460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358e5dc7-78db-4e6b-adfa-29675d6a5e49 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:11.257540 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4536368-256c-4f52-aa6f-626b23f2baae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:11.258391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4536368-256c-4f52-aa6f-626b23f2baae None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:11.258681 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4536368-256c-4f52-aa6f-626b23f2baae None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:11.258963 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4536368-256c-4f52-aa6f-626b23f2baae None admin] PATH_INFO: `/v3/users/swift` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:11.267377 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4536368-256c-4f52-aa6f-626b23f2baae None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:11.268876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4536368-256c-4f52-aa6f-626b23f2baae None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:11.285544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-e4536368-256c-4f52-aa6f-626b23f2baae None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 17:51:11.285544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:11.285544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:11.285544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:11.285544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:11.285544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:11.285544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:11.285544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:11.285544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:11.285544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:11.285544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:11.285544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:11.285544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:51:11.285544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:51:11.285544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:51:11.285544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:51:11.287376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:11.287376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:11.287376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:51:11.287376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:11.287376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:51:11.287376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:11.287376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:11.287376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:11.287376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:11.287376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:11.287376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:11.287376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:11.287376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:11.287376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:11.287376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:11.287376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:11.287376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:11.287376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:11.288548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:11.288548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:11.288548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:51:11.288548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:51:11.288548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:51:11.288548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:51:11.288548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:51:11.288548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:51:11.288548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 17:51:11.288548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:11.289385 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 345/701] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:51:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:11.290962 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4536368-256c-4f52-aa6f-626b23f2baae None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:11.365315 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358e5dc7-78db-4e6b-adfa-29675d6a5e49 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:11.391340 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:11.405604 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6993a4af-cc7b-4d21-8fee-82b5743887a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:11.406150 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4536368-256c-4f52-aa6f-626b23f2baae None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:11.406461 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993a4af-cc7b-4d21-8fee-82b5743887a0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:11.406746 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993a4af-cc7b-4d21-8fee-82b5743887a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:11.407025 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993a4af-cc7b-4d21-8fee-82b5743887a0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:11.407767 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6993a4af-cc7b-4d21-8fee-82b5743887a0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:11.409139 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6993a4af-cc7b-4d21-8fee-82b5743887a0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:11.423894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-6993a4af-cc7b-4d21-8fee-82b5743887a0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 17:51:11.423894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:11.423894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:11.423894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:11.423894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:11.423894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:11.423894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:11.423894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:11.423894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 17:51:11.423894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:11.423894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:11.423894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:11.423894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:11.423894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:11.423894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:11.423894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:11.425813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:11.425813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:11.425813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:11.425813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:11.425813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:11.425813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:11.425813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:11.425813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:11.425813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:11.425813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:11.425813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:11.425813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:11.425813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:11.425813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:11.425813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:11.425813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:11.427448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:11.427448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:11.427448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:11.427448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:11.427448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:11.427448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:11.427448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:11.427448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:11.427448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:11.427448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:11.427448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:11.427448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:11.427448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:11.427448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 17:51:11.427448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:11.427448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 357/702] 10.222.0.76 () {56 vars in 1129 bytes} [Thu May 26 17:51:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:11.429094 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103b466-956e-4b8b-8eb8-a39ce5e6fca0 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:11.429094 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103b466-956e-4b8b-8eb8-a39ce5e6fca0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:11.429094 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103b466-956e-4b8b-8eb8-a39ce5e6fca0 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:11.429544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 358/703] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:11.430706 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103b466-956e-4b8b-8eb8-a39ce5e6fca0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:11.454224 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4a3f3-490c-4eca-8deb-5d79e14ed86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:11.454987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4a3f3-490c-4eca-8deb-5d79e14ed86c None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:11.455471 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4a3f3-490c-4eca-8deb-5d79e14ed86c None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:11.455904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4a3f3-490c-4eca-8deb-5d79e14ed86c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:11.469061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:11.531272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4a3f3-490c-4eca-8deb-5d79e14ed86c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:11.532841 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4a3f3-490c-4eca-8deb-5d79e14ed86c None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:11.547701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103b466-956e-4b8b-8eb8-a39ce5e6fca0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:11.586062 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b524d-1b8a-4fa0-9d5e-00b716bc29c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:11.586768 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b524d-1b8a-4fa0-9d5e-00b716bc29c7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:11.587056 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b524d-1b8a-4fa0-9d5e-00b716bc29c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:11.587332 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b524d-1b8a-4fa0-9d5e-00b716bc29c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:11.588084 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b524d-1b8a-4fa0-9d5e-00b716bc29c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:11.588389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b524d-1b8a-4fa0-9d5e-00b716bc29c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:11.589769 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b524d-1b8a-4fa0-9d5e-00b716bc29c7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:11.598440 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 359/704] 10.222.0.76 () {56 vars in 1103 bytes} [Thu May 26 17:51:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:11.599279 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b524d-1b8a-4fa0-9d5e-00b716bc29c7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:11.647148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 346/705] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:51:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:51:11.648347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4a3f3-490c-4eca-8deb-5d79e14ed86c None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:11.655449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:11.693210 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:11.714895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b524d-1b8a-4fa0-9d5e-00b716bc29c7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:11.758902 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd624557-b3ea-44a4-bf40-1a4e6828da9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:11.759714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd624557-b3ea-44a4-bf40-1a4e6828da9c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:11.760099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd624557-b3ea-44a4-bf40-1a4e6828da9c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:11.760425 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4a3f3-490c-4eca-8deb-5d79e14ed86c None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:11.760595 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd624557-b3ea-44a4-bf40-1a4e6828da9c None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:11.761467 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd624557-b3ea-44a4-bf40-1a4e6828da9c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:11.761884 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd624557-b3ea-44a4-bf40-1a4e6828da9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:11.763945 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd624557-b3ea-44a4-bf40-1a4e6828da9c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:11.784354 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 360/706] 10.222.0.76 () {56 vars in 1135 bytes} [Thu May 26 17:51:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:11.785524 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80ccd9-b40e-4ec1-b715-df9e12046ff1 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:11.785816 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80ccd9-b40e-4ec1-b715-df9e12046ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:11.786388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80ccd9-b40e-4ec1-b715-df9e12046ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:11.804682 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ae80ccd9-b40e-4ec1-b715-df9e12046ff1 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:11.805214 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc49ad-4e45-471f-913b-04de0addf8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:11.805880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ae80ccd9-b40e-4ec1-b715-df9e12046ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:11.806020 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc49ad-4e45-471f-913b-04de0addf8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:11.806210 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc49ad-4e45-471f-913b-04de0addf8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:11.806493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc49ad-4e45-471f-913b-04de0addf8a8 None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:11.807205 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc49ad-4e45-471f-913b-04de0addf8a8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:11.807518 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc49ad-4e45-471f-913b-04de0addf8a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:11.808894 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc49ad-4e45-471f-913b-04de0addf8a8 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:11.813514 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 347/707] 10.222.0.76 () {56 vars in 1124 bytes} [Thu May 26 17:51:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:11.814347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc49ad-4e45-471f-913b-04de0addf8a8 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:11.883604 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 361/708] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:51:11.884554 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80ccd9-b40e-4ec1-b715-df9e12046ff1 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:11.923017 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:11.939377 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc49ad-4e45-471f-913b-04de0addf8a8 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:11.942808 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:11.977753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be55ac9-660c-43a9-bf33-da40092f0d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:11.978533 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be55ac9-660c-43a9-bf33-da40092f0d6e None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:11.978815 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be55ac9-660c-43a9-bf33-da40092f0d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:11.979091 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be55ac9-660c-43a9-bf33-da40092f0d6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:11.989755 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be55ac9-660c-43a9-bf33-da40092f0d6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:11.991405 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be55ac9-660c-43a9-bf33-da40092f0d6e None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:11.997759 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80ccd9-b40e-4ec1-b715-df9e12046ff1 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:12.010818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-1be55ac9-660c-43a9-bf33-da40092f0d6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:12.010818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:12.010818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:12.010818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:12.010818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:12.010818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:12.010818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:12.010818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:12.010818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:12.010818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:12.010818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:12.010818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:12.010818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:12.010818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:12.010818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:12.012723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:12.012723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:12.012723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:12.012723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:12.012723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:12.012723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:12.012723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:12.012723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:12.012723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:12.012723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:12.012723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:12.012723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:12.012723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:12.012723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:12.012723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:12.012723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:12.012723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:12.015256 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:12.015256 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:12.015256 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:12.015256 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:12.015256 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 348/709] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:12.015256 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be55ac9-660c-43a9-bf33-da40092f0d6e None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:12.038437 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bea4bb-710d-41dd-9d21-823b22f3b77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:12.039302 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bea4bb-710d-41dd-9d21-823b22f3b77d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:12.039579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bea4bb-710d-41dd-9d21-823b22f3b77d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:12.039856 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bea4bb-710d-41dd-9d21-823b22f3b77d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:12.054426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bea4bb-710d-41dd-9d21-823b22f3b77d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:12.055897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bea4bb-710d-41dd-9d21-823b22f3b77d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:12.074975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-43bea4bb-710d-41dd-9d21-823b22f3b77d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:12.074975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:12.074975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:12.074975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:12.074975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:12.074975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:12.074975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:12.074975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:12.074975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:12.074975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:12.074975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:12.074975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:12.074975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:12.074975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:12.074975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:12.076841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:12.076841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:12.076841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:12.076841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:12.076841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:12.076841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:12.076841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:12.076841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:12.076841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:12.076841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:12.076841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:12.076841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:12.076841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:12.076841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:12.076841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:12.076841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:12.076841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:12.078920 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:12.078920 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:12.078920 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:12.078920 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:12.078920 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 362/710] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:12.078920 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bea4bb-710d-41dd-9d21-823b22f3b77d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:12.131759 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be55ac9-660c-43a9-bf33-da40092f0d6e None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:12.171381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f8dbe-a3bd-40fd-b633-96d64dc57fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:12.172039 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f8dbe-a3bd-40fd-b633-96d64dc57fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:12.172317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f8dbe-a3bd-40fd-b633-96d64dc57fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:12.172598 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f8dbe-a3bd-40fd-b633-96d64dc57fb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:12.172804 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:12.186557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bea4bb-710d-41dd-9d21-823b22f3b77d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:12.211199 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:12.223629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea3da4b-2b04-4906-a388-0709f24aad46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:12.224298 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3da4b-2b04-4906-a388-0709f24aad46 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:12.224623 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3da4b-2b04-4906-a388-0709f24aad46 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:12.224898 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3da4b-2b04-4906-a388-0709f24aad46 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:12.225604 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea3da4b-2b04-4906-a388-0709f24aad46 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:12.225908 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea3da4b-2b04-4906-a388-0709f24aad46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:12.227328 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea3da4b-2b04-4906-a388-0709f24aad46 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:12.233408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 363/711] 10.222.0.76 () {56 vars in 1124 bytes} [Thu May 26 17:51:12 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:12.234137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea3da4b-2b04-4906-a388-0709f24aad46 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:12.277094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f8dbe-a3bd-40fd-b633-96d64dc57fb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:12.278772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f8dbe-a3bd-40fd-b633-96d64dc57fb4 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:12.394229 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea3da4b-2b04-4906-a388-0709f24aad46 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:12.442325 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c92e45-4bb5-4757-a2ca-74f92d022c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:12.443190 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c92e45-4bb5-4757-a2ca-74f92d022c9b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:12.443474 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c92e45-4bb5-4757-a2ca-74f92d022c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:12.443749 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c92e45-4bb5-4757-a2ca-74f92d022c9b None admin] PATH_INFO: `/v3/services/network` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:12.444440 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c92e45-4bb5-4757-a2ca-74f92d022c9b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:12.445935 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c92e45-4bb5-4757-a2ca-74f92d022c9b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:12.466893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-76c92e45-4bb5-4757-a2ca-74f92d022c9b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 17:51:12.466893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:12.466893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:12.466893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:12.466893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:12.466893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:12.466893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:12.466893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:12.466893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 17:51:12.466893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:12.466893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:12.466893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:12.466893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:12.466893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:12.466893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:12.466893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:12.466893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:12.467961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:12.467961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:12.467961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:12.467961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:12.467961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:12.467961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:12.467961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:12.467961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:12.467961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:12.467961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:12.467961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:12.467961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:12.467961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:12.467961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:12.467961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:12.467961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:12.468849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:12.468849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:12.468849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:12.468849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:12.468849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:12.468849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:12.468849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:12.468849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:12.468849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:12.468849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:12.468849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:12.468849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:12.468849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 17:51:12.468849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:12.469628 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 364/712] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:12 2022] GET /identity/v3/services/network => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:12.473674 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c92e45-4bb5-4757-a2ca-74f92d022c9b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:12.504707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:12.605581 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 349/713] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:51:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:51:12.605581 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f8dbe-a3bd-40fd-b633-96d64dc57fb4 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:12.649723 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c92e45-4bb5-4757-a2ca-74f92d022c9b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:12.690245 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:12.696840 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a2e5d-fbdb-42a9-a65d-8233bce81fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:12.697526 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a2e5d-fbdb-42a9-a65d-8233bce81fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:12.697809 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a2e5d-fbdb-42a9-a65d-8233bce81fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:12.698082 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a2e5d-fbdb-42a9-a65d-8233bce81fe2 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:12.698834 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a2e5d-fbdb-42a9-a65d-8233bce81fe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:12.699136 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a2e5d-fbdb-42a9-a65d-8233bce81fe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:12.700514 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a2e5d-fbdb-42a9-a65d-8233bce81fe2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:12.716564 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 365/714] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:12.718107 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a2e5d-fbdb-42a9-a65d-8233bce81fe2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:12.749797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f8dbe-a3bd-40fd-b633-96d64dc57fb4 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:12.794645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf048ec-a0c0-4e4b-9743-824406c4a002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:12.795222 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf048ec-a0c0-4e4b-9743-824406c4a002 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:12.795502 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf048ec-a0c0-4e4b-9743-824406c4a002 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:12.795771 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf048ec-a0c0-4e4b-9743-824406c4a002 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:12.797209 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf048ec-a0c0-4e4b-9743-824406c4a002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:12.797514 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf048ec-a0c0-4e4b-9743-824406c4a002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:12.799929 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf048ec-a0c0-4e4b-9743-824406c4a002 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:12.825685 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 350/715] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:12.825685 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf048ec-a0c0-4e4b-9743-824406c4a002 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:12.854348 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:12.891626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a2e5d-fbdb-42a9-a65d-8233bce81fe2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:12.952731 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69152c96-0bba-4fdb-bce6-93b2a3b1cc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:12.953788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69152c96-0bba-4fdb-bce6-93b2a3b1cc63 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:12.954302 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69152c96-0bba-4fdb-bce6-93b2a3b1cc63 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:12.954769 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69152c96-0bba-4fdb-bce6-93b2a3b1cc63 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:12.957254 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69152c96-0bba-4fdb-bce6-93b2a3b1cc63 None admin] RBAC: Adding query filter params (name=network) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:12.957780 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69152c96-0bba-4fdb-bce6-93b2a3b1cc63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:12.960145 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69152c96-0bba-4fdb-bce6-93b2a3b1cc63 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:12.967341 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 366/716] 10.222.0.76 () {56 vars in 1112 bytes} [Thu May 26 17:51:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:12.968682 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69152c96-0bba-4fdb-bce6-93b2a3b1cc63 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:12.986520 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf048ec-a0c0-4e4b-9743-824406c4a002 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:13.009684 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:13.030173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefd633-fe99-4a2c-aeeb-5fe370130b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:13.031008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefd633-fe99-4a2c-aeeb-5fe370130b42 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:13.031290 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefd633-fe99-4a2c-aeeb-5fe370130b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:13.031565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefd633-fe99-4a2c-aeeb-5fe370130b42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:13.045109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cefd633-fe99-4a2c-aeeb-5fe370130b42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:13.047091 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cefd633-fe99-4a2c-aeeb-5fe370130b42 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:13.067481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-5cefd633-fe99-4a2c-aeeb-5fe370130b42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:13.067481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:13.067481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:13.067481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:13.067481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:13.067481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:13.067481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:13.067481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:13.067481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:13.067481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:13.067481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:13.067481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:13.067481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:13.067481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:13.067481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:13.068441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:13.068441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:13.068441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:13.068441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:13.068441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:13.068441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:13.068441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:13.068441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:13.068441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:13.068441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:13.068441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:13.068441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:13.068441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:13.068441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:13.068441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:13.068441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:13.068441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:13.069494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:13.069494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:13.069494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:13.069494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:13.069494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 351/717] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:13.069783 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefd633-fe99-4a2c-aeeb-5fe370130b42 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:13.144914 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69152c96-0bba-4fdb-bce6-93b2a3b1cc63 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:13.190985 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45e1b0-c93b-4b55-bcf5-35a18fa2621a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:13.191835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45e1b0-c93b-4b55-bcf5-35a18fa2621a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:13.192116 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45e1b0-c93b-4b55-bcf5-35a18fa2621a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:13.192389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45e1b0-c93b-4b55-bcf5-35a18fa2621a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:13.201817 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45e1b0-c93b-4b55-bcf5-35a18fa2621a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:13.203368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45e1b0-c93b-4b55-bcf5-35a18fa2621a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:13.208049 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:13.212233 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefd633-fe99-4a2c-aeeb-5fe370130b42 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:13.224238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-eb45e1b0-c93b-4b55-bcf5-35a18fa2621a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:51:13.224238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:13.224238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:13.224238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:13.224238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:13.224238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:13.224238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:13.224238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:13.224238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:13.224238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:13.224238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:13.224238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:13.224238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:51:13.224238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:51:13.224238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:51:13.224238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:51:13.227351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:13.227351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:13.227351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:51:13.227351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:13.227351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:51:13.227351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:13.227351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:13.227351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:13.227351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:13.227351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:13.227351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:13.227351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:13.227351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:13.227351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:13.227351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:13.227351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:13.227351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:13.227351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:13.231023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:13.231023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:13.231023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:51:13.231023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:51:13.231023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:51:13.231023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:51:13.231023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:51:13.231023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:51:13.231023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:51:13.231023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:13.231023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 367/718] 10.222.0.76 () {56 vars in 1117 bytes} [Thu May 26 17:51:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:13.231023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45e1b0-c93b-4b55-bcf5-35a18fa2621a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:13.269878 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a81ac-0007-4fed-9e4d-d9c091a4ffe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:13.271128 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a81ac-0007-4fed-9e4d-d9c091a4ffe7 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:13.271544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a81ac-0007-4fed-9e4d-d9c091a4ffe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:13.272015 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a81ac-0007-4fed-9e4d-d9c091a4ffe7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:13.281542 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a81ac-0007-4fed-9e4d-d9c091a4ffe7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:13.283637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a81ac-0007-4fed-9e4d-d9c091a4ffe7 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:13.302869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-4f8a81ac-0007-4fed-9e4d-d9c091a4ffe7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:13.302869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:13.302869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:13.302869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:13.302869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:13.302869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:13.302869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:13.302869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:13.302869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:13.302869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:13.302869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:13.302869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:13.302869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:51:13.302869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:51:13.302869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:51:13.314604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:51:13.314604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:13.314604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:13.314604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:13.314604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:13.314604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:13.314604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:13.314604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:13.314604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:13.314604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:13.314604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:13.314604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:13.314604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:13.314604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:13.314604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:13.314604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:13.314604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:13.316814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:51:13.316814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:51:13.316814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:51:13.316814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:51:13.316814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:51:13.316814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:51:13.316814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:13.316814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:13.316814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 352/719] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:13.316814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a81ac-0007-4fed-9e4d-d9c091a4ffe7 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:13.362879 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:13.389845 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45e1b0-c93b-4b55-bcf5-35a18fa2621a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:13.436074 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c01350-04d4-48ba-8c98-05de9a31baf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:13.436745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c01350-04d4-48ba-8c98-05de9a31baf8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:13.437052 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c01350-04d4-48ba-8c98-05de9a31baf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:13.437320 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c01350-04d4-48ba-8c98-05de9a31baf8 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:13.438034 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c01350-04d4-48ba-8c98-05de9a31baf8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:13.438371 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c01350-04d4-48ba-8c98-05de9a31baf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:13.440237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c01350-04d4-48ba-8c98-05de9a31baf8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:13.453631 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 368/720] 10.222.0.76 () {56 vars in 1112 bytes} [Thu May 26 17:51:13 2022] GET /identity/v3/services?type=network => generated 124 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:13.453631 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c01350-04d4-48ba-8c98-05de9a31baf8 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:13.498733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a81ac-0007-4fed-9e4d-d9c091a4ffe7 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:13.544422 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:13.574776 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3cfe1-666d-438c-9a78-a3d91e86c6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:13.575746 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3cfe1-666d-438c-9a78-a3d91e86c6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:13.576226 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3cfe1-666d-438c-9a78-a3d91e86c6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:13.576701 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3cfe1-666d-438c-9a78-a3d91e86c6c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:13.578015 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3cfe1-666d-438c-9a78-a3d91e86c6c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:13.582509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3cfe1-666d-438c-9a78-a3d91e86c6c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:13.584110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3cfe1-666d-438c-9a78-a3d91e86c6c0 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:13.609005 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 353/721] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:13.610620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3cfe1-666d-438c-9a78-a3d91e86c6c0 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:13.645483 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c01350-04d4-48ba-8c98-05de9a31baf8 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:13.711566 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723730fa-cd90-467b-8ad0-902dba0e13f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:13.712258 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723730fa-cd90-467b-8ad0-902dba0e13f2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:13.712535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723730fa-cd90-467b-8ad0-902dba0e13f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:13.712815 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723730fa-cd90-467b-8ad0-902dba0e13f2 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:13.714381 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723730fa-cd90-467b-8ad0-902dba0e13f2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:13.714704 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723730fa-cd90-467b-8ad0-902dba0e13f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:13.716508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723730fa-cd90-467b-8ad0-902dba0e13f2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:13.756274 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 369/722] 10.222.0.76 () {56 vars in 1149 bytes} [Thu May 26 17:51:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:13.757857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723730fa-cd90-467b-8ad0-902dba0e13f2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:13.786357 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:13.810059 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3cfe1-666d-438c-9a78-a3d91e86c6c0 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:13.875904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada1e3c-01ea-4975-80d1-ebb07bfd7165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:13.877080 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada1e3c-01ea-4975-80d1-ebb07bfd7165 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:13.878684 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada1e3c-01ea-4975-80d1-ebb07bfd7165 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:13.879289 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada1e3c-01ea-4975-80d1-ebb07bfd7165 None admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:13.880836 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada1e3c-01ea-4975-80d1-ebb07bfd7165 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:13.881854 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada1e3c-01ea-4975-80d1-ebb07bfd7165 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:13.884275 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada1e3c-01ea-4975-80d1-ebb07bfd7165 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:13.913626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 354/723] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:51:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:13.913626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada1e3c-01ea-4975-80d1-ebb07bfd7165 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:13.970356 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723730fa-cd90-467b-8ad0-902dba0e13f2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:14.014825 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:14.033594 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7444f-9110-4e1a-a8c0-42676637b32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:14.034445 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7444f-9110-4e1a-a8c0-42676637b32f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:14.034734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7444f-9110-4e1a-a8c0-42676637b32f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:14.035039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7444f-9110-4e1a-a8c0-42676637b32f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:14.043960 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d7444f-9110-4e1a-a8c0-42676637b32f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:14.045498 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d7444f-9110-4e1a-a8c0-42676637b32f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:14.061330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-f6d7444f-9110-4e1a-a8c0-42676637b32f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:14.061330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:14.061330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:14.061330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:14.061330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:14.061330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:14.061330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:14.061330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:14.061330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:14.061330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:14.061330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:14.061330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:14.061330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:51:14.061330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:51:14.061330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:51:14.063104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:51:14.063104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:14.063104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:14.063104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:14.063104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:14.063104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:14.063104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:14.063104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:14.063104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:14.063104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:14.063104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:14.063104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:14.063104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:14.063104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:14.063104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:14.063104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:14.063104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:14.066263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:51:14.066263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:51:14.066263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:51:14.066263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:51:14.066263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:51:14.066263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:51:14.066263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:14.066263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:14.069494 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 370/724] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:14.069494 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7444f-9110-4e1a-a8c0-42676637b32f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:14.104203 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada1e3c-01ea-4975-80d1-ebb07bfd7165 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:14.183191 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84c21a-a3d6-4970-b0da-a384f71c7352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:14.184827 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84c21a-a3d6-4970-b0da-a384f71c7352 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:14.185840 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84c21a-a3d6-4970-b0da-a384f71c7352 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:14.186948 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84c21a-a3d6-4970-b0da-a384f71c7352 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:14.193445 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7444f-9110-4e1a-a8c0-42676637b32f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:14.243775 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:14.255462 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221dcb29-cacb-4361-a8d1-369ff437e799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:14.258546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221dcb29-cacb-4361-a8d1-369ff437e799 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:14.259999 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221dcb29-cacb-4361-a8d1-369ff437e799 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:14.261299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221dcb29-cacb-4361-a8d1-369ff437e799 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:14.264692 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221dcb29-cacb-4361-a8d1-369ff437e799 None admin] RBAC: Adding query filter params (role.id=4fa8f06085c741a5b3cf64e63cc0754f, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=1899c317083d40f5b490cd9eb63fe521) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:14.266389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221dcb29-cacb-4361-a8d1-369ff437e799 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:14.270567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221dcb29-cacb-4361-a8d1-369ff437e799 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:14.281075 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 371/725] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:51:14 2022] GET /identity/v3/role_assignments?user.id=1899c317083d40f5b490cd9eb63fe521&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=4fa8f06085c741a5b3cf64e63cc0754f => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:14.283172 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221dcb29-cacb-4361-a8d1-369ff437e799 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:14.320649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a84c21a-a3d6-4970-b0da-a384f71c7352 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:14.322634 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a84c21a-a3d6-4970-b0da-a384f71c7352 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:14.377417 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:14.453674 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 355/726] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:51:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 17:51:14.462518 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84c21a-a3d6-4970-b0da-a384f71c7352 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:14.492392 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221dcb29-cacb-4361-a8d1-369ff437e799 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:14.558214 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf9e02-286f-4b72-b3a3-477d62643627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:14.559251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf9e02-286f-4b72-b3a3-477d62643627 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:14.559694 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf9e02-286f-4b72-b3a3-477d62643627 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:14.560151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf9e02-286f-4b72-b3a3-477d62643627 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:14.561544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf9e02-286f-4b72-b3a3-477d62643627 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:14.562060 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf9e02-286f-4b72-b3a3-477d62643627 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:14.565475 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf9e02-286f-4b72-b3a3-477d62643627 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:14.582725 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:14.593241 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 372/727] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:51:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:14.604881 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf9e02-286f-4b72-b3a3-477d62643627 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:14.696207 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84c21a-a3d6-4970-b0da-a384f71c7352 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:14.738530 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac3498-5e1c-4f0e-bdc1-d1a723d73f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:14.739931 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac3498-5e1c-4f0e-bdc1-d1a723d73f3e None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:14.740426 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac3498-5e1c-4f0e-bdc1-d1a723d73f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:14.740914 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac3498-5e1c-4f0e-bdc1-d1a723d73f3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:14.759548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac3498-5e1c-4f0e-bdc1-d1a723d73f3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:14.761681 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac3498-5e1c-4f0e-bdc1-d1a723d73f3e None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:14.785090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-adac3498-5e1c-4f0e-bdc1-d1a723d73f3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:14.785090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:14.785090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:14.785090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:14.785090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:14.785090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:14.785090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:14.785090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:14.785090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:14.785090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:14.785090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:14.785090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:14.785090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:14.785090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:14.785090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:14.786344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:14.786344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:14.786344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:14.786344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:14.786344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:14.786344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:14.786344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:14.786344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:14.786344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:14.786344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:14.786344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:14.786344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:14.786344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:14.786344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:14.786344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:14.786344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:14.786344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:14.787291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:14.787291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:14.787291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:14.787291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:14.794348 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 356/728] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:14.796407 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf9e02-286f-4b72-b3a3-477d62643627 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:14.798826 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c1e36-410e-45b3-9943-e2df7fb6becb None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:14.800049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c1e36-410e-45b3-9943-e2df7fb6becb None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:14.801177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c1e36-410e-45b3-9943-e2df7fb6becb None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:14.804275 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 357/729] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:14.805872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c1e36-410e-45b3-9943-e2df7fb6becb None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:14.807035 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:14.841938 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453201b-3c4a-4f4c-a76d-bcefcb8240e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:14.843041 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453201b-3c4a-4f4c-a76d-bcefcb8240e1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:14.843532 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453201b-3c4a-4f4c-a76d-bcefcb8240e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:14.844025 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453201b-3c4a-4f4c-a76d-bcefcb8240e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:14.845804 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9453201b-3c4a-4f4c-a76d-bcefcb8240e1 None admin] RBAC: Adding query filter params (role.id=e776303775a2423aa0c62c161ecb5a7e, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=9ca774e864ca44cfb187e38bc8a8f6ea) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:14.846896 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9453201b-3c4a-4f4c-a76d-bcefcb8240e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:14.848875 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9453201b-3c4a-4f4c-a76d-bcefcb8240e1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:14.857818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 373/730] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:51:14 2022] GET /identity/v3/role_assignments?user.id=9ca774e864ca44cfb187e38bc8a8f6ea&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=e776303775a2423aa0c62c161ecb5a7e => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:14.859446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d813a96-c9ff-4d1d-a7b9-e41d064a7445 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:14.859921 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d813a96-c9ff-4d1d-a7b9-e41d064a7445 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:14.860613 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d813a96-c9ff-4d1d-a7b9-e41d064a7445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:14.888308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0d813a96-c9ff-4d1d-a7b9-e41d064a7445 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:14.890248 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0d813a96-c9ff-4d1d-a7b9-e41d064a7445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:14.988038 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:14.990358 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c1e36-410e-45b3-9943-e2df7fb6becb None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:15.053955 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 374/731] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:51:15.070438 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7bfd62-900a-425d-b349-915495fd47ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:15.071287 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7bfd62-900a-425d-b349-915495fd47ac None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:15.071578 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7bfd62-900a-425d-b349-915495fd47ac None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:15.071902 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7bfd62-900a-425d-b349-915495fd47ac None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:15.072918 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7bfd62-900a-425d-b349-915495fd47ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:15.073299 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7bfd62-900a-425d-b349-915495fd47ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:15.074963 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7bfd62-900a-425d-b349-915495fd47ac None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:15.079325 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d813a96-c9ff-4d1d-a7b9-e41d064a7445 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:15.099998 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 358/732] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:15.112725 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7bfd62-900a-425d-b349-915495fd47ac None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:15.255807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:15.262340 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d813a96-c9ff-4d1d-a7b9-e41d064a7445 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:15.301485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7bfd62-900a-425d-b349-915495fd47ac None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:15.342478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b04c32-81aa-4042-810d-379debd99810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:15.343897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b04c32-81aa-4042-810d-379debd99810 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:15.344398 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b04c32-81aa-4042-810d-379debd99810 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:15.345373 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b04c32-81aa-4042-810d-379debd99810 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:15.347313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b04c32-81aa-4042-810d-379debd99810 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:15.350356 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b04c32-81aa-4042-810d-379debd99810 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:15.360156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27db54-9778-47f0-a775-93a681a5ea71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:15.363191 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27db54-9778-47f0-a775-93a681a5ea71 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:15.363878 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27db54-9778-47f0-a775-93a681a5ea71 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:15.364508 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27db54-9778-47f0-a775-93a681a5ea71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:15.376869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27db54-9778-47f0-a775-93a681a5ea71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:15.378170 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 375/733] 10.222.0.76 () {60 vars in 1141 bytes} [Thu May 26 17:51:15 2022] POST /identity/v3/services => generated 251 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:51:15.385150 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27db54-9778-47f0-a775-93a681a5ea71 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:15.412253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-ab27db54-9778-47f0-a775-93a681a5ea71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:15.412253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:15.412253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:15.412253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:15.412253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:15.412253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:15.412253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:15.412253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:15.412253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:15.412253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:15.412253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:15.412253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:15.412253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:51:15.412253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:51:15.412253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:51:15.414047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:51:15.414047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:15.414047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:15.414047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:15.414047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:15.414047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:15.414047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:15.414047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:15.414047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:15.414047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:15.414047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:15.414047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:15.414047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:15.414047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:15.414047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:15.414047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:15.414047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:15.416540 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:51:15.416540 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:51:15.416540 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:51:15.416540 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:51:15.416540 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:51:15.416540 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:51:15.416540 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:15.416540 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:15.416540 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 359/734] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:15.427240 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b04c32-81aa-4042-810d-379debd99810 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:15.647418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b04c32-81aa-4042-810d-379debd99810 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:15.703128 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276ba9d-c7cc-4f3e-b072-aa372feec827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:15.704681 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276ba9d-c7cc-4f3e-b072-aa372feec827 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:15.705149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276ba9d-c7cc-4f3e-b072-aa372feec827 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:15.705654 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276ba9d-c7cc-4f3e-b072-aa372feec827 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:15.707380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e276ba9d-c7cc-4f3e-b072-aa372feec827 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:15.707850 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e276ba9d-c7cc-4f3e-b072-aa372feec827 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:15.710146 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e276ba9d-c7cc-4f3e-b072-aa372feec827 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:15.734393 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 376/735] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:51:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:15.740714 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27db54-9778-47f0-a775-93a681a5ea71 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:15.783948 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:15.927542 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27db54-9778-47f0-a775-93a681a5ea71 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:15.969587 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc914240-521d-4a2b-b77b-f9c9d3339b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:15.970441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc914240-521d-4a2b-b77b-f9c9d3339b24 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:15.970739 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc914240-521d-4a2b-b77b-f9c9d3339b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:15.971643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc914240-521d-4a2b-b77b-f9c9d3339b24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:15.972846 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc914240-521d-4a2b-b77b-f9c9d3339b24 None admin] RBAC: Adding query filter params (role.id=e776303775a2423aa0c62c161ecb5a7e, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=3e06d1b0e1a9472db764f4be7e42fdd9) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:15.973178 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc914240-521d-4a2b-b77b-f9c9d3339b24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:15.974943 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc914240-521d-4a2b-b77b-f9c9d3339b24 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:15.986815 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 360/736] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:51:15 2022] GET /identity/v3/role_assignments?user.id=3e06d1b0e1a9472db764f4be7e42fdd9&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=e776303775a2423aa0c62c161ecb5a7e => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:16.779815 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb081db8-ba71-42a7-883d-d0357dd40d62 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:16.780379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb081db8-ba71-42a7-883d-d0357dd40d62 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:16.780379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb081db8-ba71-42a7-883d-d0357dd40d62 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:16.782019 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 377/737] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:16.788878 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474a393-e29e-4a72-82ea-35509bf984ea None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:16.789387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474a393-e29e-4a72-82ea-35509bf984ea None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:16.789494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474a393-e29e-4a72-82ea-35509bf984ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:16.823962 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-3474a393-e29e-4a72-82ea-35509bf984ea None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:16.825202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-3474a393-e29e-4a72-82ea-35509bf984ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:16.945633 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 361/738] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:16 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 17:51:16.980868 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb081db8-ba71-42a7-883d-d0357dd40d62 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:17.012536 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:17.156154 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb081db8-ba71-42a7-883d-d0357dd40d62 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:17.209363 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521d630-113e-45bb-b9aa-0c711805c057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:17.210460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521d630-113e-45bb-b9aa-0c711805c057 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:17.210866 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521d630-113e-45bb-b9aa-0c711805c057 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:17.211451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521d630-113e-45bb-b9aa-0c711805c057 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:17.212636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e521d630-113e-45bb-b9aa-0c711805c057 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:17.215190 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e521d630-113e-45bb-b9aa-0c711805c057 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:17.251765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 378/739] 10.222.0.76 () {60 vars in 1141 bytes} [Thu May 26 17:51:16 2022] POST /identity/v3/services => generated 240 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:51:17.640490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bd52c-14fa-4ea7-9e8c-bb9e5681df06 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:17.640977 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bd52c-14fa-4ea7-9e8c-bb9e5681df06 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:17.641105 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bd52c-14fa-4ea7-9e8c-bb9e5681df06 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:17.642547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 362/740] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:17.650914 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296165e2-25c4-4dcd-9bba-fe85507ff2fa None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:17.651242 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296165e2-25c4-4dcd-9bba-fe85507ff2fa None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:17.651578 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296165e2-25c4-4dcd-9bba-fe85507ff2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:17.675680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-296165e2-25c4-4dcd-9bba-fe85507ff2fa None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:17.677004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-296165e2-25c4-4dcd-9bba-fe85507ff2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:17.776981 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 379/741] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:17.804868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09bd52c-14fa-4ea7-9e8c-bb9e5681df06 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:17.826753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:17.951754 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09bd52c-14fa-4ea7-9e8c-bb9e5681df06 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:18.005996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a72e0-f0da-491d-8801-d164351c14ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:18.007016 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a72e0-f0da-491d-8801-d164351c14ca None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:18.007405 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a72e0-f0da-491d-8801-d164351c14ca None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:18.007694 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a72e0-f0da-491d-8801-d164351c14ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:18.027108 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a72e0-f0da-491d-8801-d164351c14ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:51:18.036956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-e79a72e0-f0da-491d-8801-d164351c14ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:51:18.036956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:18.036956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:18.036956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:18.036956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:18.036956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:18.036956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:18.036956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:18.036956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:18.036956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:18.036956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:18.036956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:18.036956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:51:18.036956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:51:18.036956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:51:18.036956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:51:18.038297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:51:18.038297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:51:18.038297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:51:18.038297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:51:18.038297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:18.038297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:18.038297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:18.038297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:18.038297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:18.038297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:18.038297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:18.038297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:18.038297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:18.038297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:18.038297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:18.038297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:18.039431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:18.039431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:18.039431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:18.039431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:18.039431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:51:18.039431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:51:18.039431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:51:18.039431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:51:18.039431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:51:18.039431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:51:18.039431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:51:18.039431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:18.039431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 363/742] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:51:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:18.045772 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296165e2-25c4-4dcd-9bba-fe85507ff2fa None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:18.048802 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:18.234171 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296165e2-25c4-4dcd-9bba-fe85507ff2fa None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:18.238955 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9652a3-9211-44ca-a724-4f88585442b9 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:18.239227 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9652a3-9211-44ca-a724-4f88585442b9 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:18.239521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9652a3-9211-44ca-a724-4f88585442b9 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:18.241330 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 364/743] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:18.248014 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa4f8c-3c73-4060-9cce-408defc820a4 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:18.248390 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa4f8c-3c73-4060-9cce-408defc820a4 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:18.248680 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa4f8c-3c73-4060-9cce-408defc820a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:18.284751 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-f1fa4f8c-3c73-4060-9cce-408defc820a4 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:18.286601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f1fa4f8c-3c73-4060-9cce-408defc820a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:18.294452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450873b-9dff-490f-a0d9-5ecd5f7b6c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:18.295901 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450873b-9dff-490f-a0d9-5ecd5f7b6c07 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:18.298369 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450873b-9dff-490f-a0d9-5ecd5f7b6c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:18.298889 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450873b-9dff-490f-a0d9-5ecd5f7b6c07 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:18.300107 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450873b-9dff-490f-a0d9-5ecd5f7b6c07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:18.301600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450873b-9dff-490f-a0d9-5ecd5f7b6c07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:18.303401 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450873b-9dff-490f-a0d9-5ecd5f7b6c07 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:18.321653 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 380/744] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:51:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:18.328668 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450873b-9dff-490f-a0d9-5ecd5f7b6c07 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:18.420096 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 365/745] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:51:18.453803 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:18.456945 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa4f8c-3c73-4060-9cce-408defc820a4 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:18.460616 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:18.555975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450873b-9dff-490f-a0d9-5ecd5f7b6c07 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:18.599072 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab9f46-b80c-47c0-9a16-c36b6f184c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:18.600181 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab9f46-b80c-47c0-9a16-c36b6f184c96 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:18.600627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab9f46-b80c-47c0-9a16-c36b6f184c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:18.601056 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab9f46-b80c-47c0-9a16-c36b6f184c96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:18.602136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa4f8c-3c73-4060-9cce-408defc820a4 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:18.615311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab9f46-b80c-47c0-9a16-c36b6f184c96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:18.617227 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab9f46-b80c-47c0-9a16-c36b6f184c96 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:18.637127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c7ab9f46-b80c-47c0-9a16-c36b6f184c96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:18.637127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:18.637127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:18.637127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:18.637127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:18.637127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:18.637127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:18.637127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:18.637127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:18.637127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:18.637127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:18.637127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:18.637127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:18.637127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:18.637127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:18.638382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:18.638382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:18.638382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:18.638382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:18.638382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:18.638382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:18.638382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:18.638382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:18.638382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:18.638382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:18.638382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:18.638382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:18.638382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:18.638382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:18.638382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:18.638382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:18.638382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:18.639420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:18.639420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:18.639420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:18.639420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:18.645801 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 381/746] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:18.647055 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152f8b0-4ae0-4d88-987b-7dfc4ef72f92 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:18.647368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152f8b0-4ae0-4d88-987b-7dfc4ef72f92 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:18.647686 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152f8b0-4ae0-4d88-987b-7dfc4ef72f92 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:18.649306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 382/747] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:18.652207 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152f8b0-4ae0-4d88-987b-7dfc4ef72f92 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:18.654108 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85124f-9141-41b6-9307-5403c4f3b7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:18.654764 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85124f-9141-41b6-9307-5403c4f3b7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:18.655152 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85124f-9141-41b6-9307-5403c4f3b7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:18.655470 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85124f-9141-41b6-9307-5403c4f3b7c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:18.670488 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85124f-9141-41b6-9307-5403c4f3b7c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:51:18.680579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-5f85124f-9141-41b6-9307-5403c4f3b7c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:51:18.680579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:18.680579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:18.680579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:18.680579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:18.680579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:18.680579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:18.680579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:18.680579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:18.680579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:18.680579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:18.680579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:18.680579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:51:18.680579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:51:18.680579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:51:18.680579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:51:18.681653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:51:18.681653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:51:18.681653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:51:18.681653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:51:18.681653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:18.681653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:18.681653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:18.681653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:18.681653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:18.681653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:18.681653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:18.681653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:18.681653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:18.681653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:18.681653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:18.681653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:18.687944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:18.687944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:18.687944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:18.687944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:18.687944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:51:18.687944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:51:18.687944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:51:18.687944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:51:18.687944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:51:18.687944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:51:18.687944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:51:18.687944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:18.687944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 366/748] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:51:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:18.687944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389b831-cb0b-403a-abf3-19b08b10c2a6 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:18.688787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389b831-cb0b-403a-abf3-19b08b10c2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:18.688787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389b831-cb0b-403a-abf3-19b08b10c2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:18.706422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-4389b831-cb0b-403a-abf3-19b08b10c2a6 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:18.707784 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-4389b831-cb0b-403a-abf3-19b08b10c2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:18.777202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:18.794095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 367/749] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:18.795408 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389b831-cb0b-403a-abf3-19b08b10c2a6 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:18.797060 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152f8b0-4ae0-4d88-987b-7dfc4ef72f92 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:18.816690 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:18.844674 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697caa9-c772-4311-aa2a-9e1f407ddc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:18.846644 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697caa9-c772-4311-aa2a-9e1f407ddc1a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:18.847281 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697caa9-c772-4311-aa2a-9e1f407ddc1a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:18.847865 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697caa9-c772-4311-aa2a-9e1f407ddc1a None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:18.848914 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1697caa9-c772-4311-aa2a-9e1f407ddc1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:18.849486 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1697caa9-c772-4311-aa2a-9e1f407ddc1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:18.851305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1697caa9-c772-4311-aa2a-9e1f407ddc1a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:18.870012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 383/750] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:18.876678 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697caa9-c772-4311-aa2a-9e1f407ddc1a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:18.996424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389b831-cb0b-403a-abf3-19b08b10c2a6 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:19.034650 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697caa9-c772-4311-aa2a-9e1f407ddc1a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:19.063709 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ffb1c-78b5-4a9c-b5b8-435728c46d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:19.066272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ffb1c-78b5-4a9c-b5b8-435728c46d0f None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:19.066817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ffb1c-78b5-4a9c-b5b8-435728c46d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:19.067204 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ffb1c-78b5-4a9c-b5b8-435728c46d0f None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:19.068546 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ffb1c-78b5-4a9c-b5b8-435728c46d0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:19.069053 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ffb1c-78b5-4a9c-b5b8-435728c46d0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:19.072123 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ffb1c-78b5-4a9c-b5b8-435728c46d0f None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:19.080116 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b9af0-b314-4e40-840a-38f8be27ca8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:19.081244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b9af0-b314-4e40-840a-38f8be27ca8f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:19.081552 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b9af0-b314-4e40-840a-38f8be27ca8f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:19.081831 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b9af0-b314-4e40-840a-38f8be27ca8f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:19.082609 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b9af0-b314-4e40-840a-38f8be27ca8f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:19.084202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b9af0-b314-4e40-840a-38f8be27ca8f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:19.093029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 368/751] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:51:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:19.093029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ffb1c-78b5-4a9c-b5b8-435728c46d0f None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:19.101253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-e87b9af0-b314-4e40-840a-38f8be27ca8f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 17:51:19.101253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:19.101253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:19.101253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:19.101253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:19.101253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:19.101253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:19.101253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:19.101253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 17:51:19.101253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:19.101253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:19.101253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:19.101253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:19.101253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:19.101253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:19.101253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:19.103699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:19.103699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:19.103699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:19.103699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:19.103699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:19.103699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:19.103699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:19.103699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:19.103699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:19.103699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:19.103699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:19.103699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:19.103699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:19.103699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:19.103699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:19.103699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:19.105376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:19.105376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:19.105376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:19.105376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:19.105376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:19.105376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:19.105376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:19.105376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:19.105376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:19.105376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:19.105376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:19.105376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:19.105376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:19.105376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 17:51:19.105376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:19.105376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 384/752] 10.222.0.76 () {56 vars in 1129 bytes} [Thu May 26 17:51:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:19.108219 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5224ec5-e023-4131-aa12-9db66064d6ba None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:19.108219 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5224ec5-e023-4131-aa12-9db66064d6ba None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:19.108219 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5224ec5-e023-4131-aa12-9db66064d6ba None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:19.108219 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 385/753] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:19.108219 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5224ec5-e023-4131-aa12-9db66064d6ba None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:19.172702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:19.184722 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:19.253234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ffb1c-78b5-4a9c-b5b8-435728c46d0f None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:19.269200 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5224ec5-e023-4131-aa12-9db66064d6ba None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:19.317014 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0f82f-019d-440e-83e4-46e140c47514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:19.317827 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0f82f-019d-440e-83e4-46e140c47514 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:19.318087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0f82f-019d-440e-83e4-46e140c47514 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:19.318435 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0f82f-019d-440e-83e4-46e140c47514 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:19.322999 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e216258-07cb-492a-a46f-ae5a41994444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:19.323861 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e216258-07cb-492a-a46f-ae5a41994444 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:19.324145 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e216258-07cb-492a-a46f-ae5a41994444 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:19.324423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e216258-07cb-492a-a46f-ae5a41994444 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:19.337963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e216258-07cb-492a-a46f-ae5a41994444 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:19.339742 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e216258-07cb-492a-a46f-ae5a41994444 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:19.341447 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa0f82f-019d-440e-83e4-46e140c47514 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:19.344242 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa0f82f-019d-440e-83e4-46e140c47514 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:19.362848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-0fa0f82f-019d-440e-83e4-46e140c47514 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 17:51:19.362848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:19.362848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:19.362848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:19.362848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:19.362848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:19.362848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:19.362848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:19.362848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:19.362848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:19.362848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:19.362848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:19.362848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:51:19.362848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:51:19.362848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:51:19.362848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:51:19.364429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:19.364429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:19.364429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:51:19.364429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:19.364429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:51:19.364429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:19.364429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:19.364429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:19.364429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:19.364429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:19.364429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:19.364429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:19.364429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:19.364429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:19.364429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:19.364429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:19.364429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:19.364429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:19.366229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:19.366229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:19.366229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:51:19.366229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:51:19.366229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:51:19.366229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:51:19.366229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:51:19.366229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:51:19.366229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 17:51:19.366229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:19.367104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-3e216258-07cb-492a-a46f-ae5a41994444 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:19.367104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:19.367104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:19.367104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:19.367104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:19.367104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:19.367104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:19.367104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:19.367104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:19.367104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:19.367104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:19.367104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:19.367104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:19.367104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:19.367104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:19.368292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:19.368292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:19.368292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:19.368292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:19.368292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:19.368292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:19.368292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:19.368292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:19.368292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:19.368292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:19.368292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:19.368292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:19.368292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:19.368292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:19.368292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:19.368292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:19.368292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:19.369348 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:19.369348 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:19.369348 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:19.369348 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:19.369700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 369/754] 10.222.0.76 () {56 vars in 1093 bytes} [Thu May 26 17:51:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:19.371396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0f82f-019d-440e-83e4-46e140c47514 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:19.372020 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 386/755] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:19.373050 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dae725-9f5d-40ce-9035-825d0a4dbf60 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:19.373366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dae725-9f5d-40ce-9035-825d0a4dbf60 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:19.373620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dae725-9f5d-40ce-9035-825d0a4dbf60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:19.399751 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c1dae725-9f5d-40ce-9035-825d0a4dbf60 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:19.402405 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c1dae725-9f5d-40ce-9035-825d0a4dbf60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:19.504077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 387/756] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:19.506715 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dae725-9f5d-40ce-9035-825d0a4dbf60 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:19.532387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0f82f-019d-440e-83e4-46e140c47514 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:19.551703 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:19.562256 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:19.591242 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f540e-0a09-499e-9972-217a12146866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:19.591930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f540e-0a09-499e-9972-217a12146866 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:19.592279 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f540e-0a09-499e-9972-217a12146866 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:19.592703 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f540e-0a09-499e-9972-217a12146866 None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:19.593520 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f540e-0a09-499e-9972-217a12146866 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:19.593960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f540e-0a09-499e-9972-217a12146866 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:19.595797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f540e-0a09-499e-9972-217a12146866 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:19.601768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 370/757] 10.222.0.76 () {56 vars in 1124 bytes} [Thu May 26 17:51:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:19.602646 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f540e-0a09-499e-9972-217a12146866 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:19.681705 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dae725-9f5d-40ce-9035-825d0a4dbf60 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:19.726831 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a61f3-5406-44be-b3c6-587ea9d63c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:19.727821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a61f3-5406-44be-b3c6-587ea9d63c92 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:19.728257 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a61f3-5406-44be-b3c6-587ea9d63c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:19.728699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a61f3-5406-44be-b3c6-587ea9d63c92 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:19.729701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a61f3-5406-44be-b3c6-587ea9d63c92 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:19.730165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a61f3-5406-44be-b3c6-587ea9d63c92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:19.731816 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a61f3-5406-44be-b3c6-587ea9d63c92 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:19.747955 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f540e-0a09-499e-9972-217a12146866 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:19.756374 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 388/758] 10.222.0.76 () {56 vars in 1133 bytes} [Thu May 26 17:51:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:19.758303 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a61f3-5406-44be-b3c6-587ea9d63c92 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:19.808858 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e883fe-cb4c-4512-b8b1-1b45f1bfc99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:19.809552 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e883fe-cb4c-4512-b8b1-1b45f1bfc99b None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:19.809859 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e883fe-cb4c-4512-b8b1-1b45f1bfc99b None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:19.810205 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e883fe-cb4c-4512-b8b1-1b45f1bfc99b None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:19.810934 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e883fe-cb4c-4512-b8b1-1b45f1bfc99b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:19.811243 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e883fe-cb4c-4512-b8b1-1b45f1bfc99b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:19.812678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e883fe-cb4c-4512-b8b1-1b45f1bfc99b None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:19.827513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 371/759] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:19.828229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e883fe-cb4c-4512-b8b1-1b45f1bfc99b None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:19.884382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:19.898021 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:19.918714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a61f3-5406-44be-b3c6-587ea9d63c92 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:19.972678 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e1567-3ab6-4861-9945-82fea7b3ff83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:19.973967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e1567-3ab6-4861-9945-82fea7b3ff83 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:19.974491 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e1567-3ab6-4861-9945-82fea7b3ff83 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:19.974998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e1567-3ab6-4861-9945-82fea7b3ff83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:19.982832 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e883fe-cb4c-4512-b8b1-1b45f1bfc99b None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:19.995278 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e1567-3ab6-4861-9945-82fea7b3ff83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:51:20.010836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-359e1567-3ab6-4861-9945-82fea7b3ff83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:51:20.010836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:20.010836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:20.010836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:20.010836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:20.010836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:20.010836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:20.010836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:20.010836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:20.010836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:20.010836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:20.010836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:20.010836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:51:20.010836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:51:20.010836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:51:20.010836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:51:20.012073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:51:20.012073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:51:20.012073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:51:20.012073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:51:20.012073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:20.012073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:20.012073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:20.012073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:20.012073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:20.012073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:20.012073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:20.012073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:20.012073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:20.012073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:20.012073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:20.012073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:20.013416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:20.013416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:20.013416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:20.013416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:20.013416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:51:20.013416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:51:20.013416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:51:20.013416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:51:20.013416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:51:20.013416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:51:20.013416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:51:20.013416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:20.013416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 389/760] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:51:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:20.015098 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e1567-3ab6-4861-9945-82fea7b3ff83 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:20.038629 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc286cd-0a3c-4060-a6aa-9bb57f4d1cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:20.039756 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc286cd-0a3c-4060-a6aa-9bb57f4d1cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:20.040278 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc286cd-0a3c-4060-a6aa-9bb57f4d1cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:20.040778 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc286cd-0a3c-4060-a6aa-9bb57f4d1cf5 None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:20.042138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc286cd-0a3c-4060-a6aa-9bb57f4d1cf5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:20.042690 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc286cd-0a3c-4060-a6aa-9bb57f4d1cf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:20.044932 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc286cd-0a3c-4060-a6aa-9bb57f4d1cf5 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:20.055444 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 372/761] 10.222.0.76 () {56 vars in 1124 bytes} [Thu May 26 17:51:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:20.062139 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc286cd-0a3c-4060-a6aa-9bb57f4d1cf5 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:20.162176 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e1567-3ab6-4861-9945-82fea7b3ff83 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:20.213962 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc63fc-0adf-48dc-bfe3-015d43f3cf29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:20.215254 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc63fc-0adf-48dc-bfe3-015d43f3cf29 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:20.215254 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc63fc-0adf-48dc-bfe3-015d43f3cf29 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:20.215567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc63fc-0adf-48dc-bfe3-015d43f3cf29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:20.232929 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:20.235881 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc63fc-0adf-48dc-bfe3-015d43f3cf29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:20.238130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc63fc-0adf-48dc-bfe3-015d43f3cf29 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:20.254075 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc286cd-0a3c-4060-a6aa-9bb57f4d1cf5 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:20.259071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-8efc63fc-0adf-48dc-bfe3-015d43f3cf29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:20.259071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:20.259071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:20.259071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:20.259071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:20.259071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:20.259071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:20.259071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:20.259071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:20.259071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:20.259071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:20.259071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:20.259071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:20.259071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:20.259071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:20.274927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:20.274927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:20.274927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:20.274927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:20.274927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:20.274927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:20.274927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:20.274927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:20.274927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:20.274927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:20.274927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:20.274927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:20.274927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:20.274927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:20.274927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:20.274927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:20.274927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:20.276702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:20.276702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:20.276702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:20.276702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:20.277139 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:20.277276 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 390/762] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:20.277276 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc63fc-0adf-48dc-bfe3-015d43f3cf29 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:20.326634 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d4870-cd37-4efa-8363-0a6b0c855307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:20.327793 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d4870-cd37-4efa-8363-0a6b0c855307 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:20.328135 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d4870-cd37-4efa-8363-0a6b0c855307 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:20.328673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d4870-cd37-4efa-8363-0a6b0c855307 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:20.344066 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d4870-cd37-4efa-8363-0a6b0c855307 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:20.347269 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d4870-cd37-4efa-8363-0a6b0c855307 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:20.375259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-032d4870-cd37-4efa-8363-0a6b0c855307 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 17:51:20.375259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:20.375259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:20.375259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:20.375259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:20.375259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:20.375259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:20.375259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:20.375259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:20.375259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:20.375259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:20.375259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:20.375259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:51:20.375259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:51:20.375259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:51:20.375259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:51:20.376274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:20.376274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:20.376274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:51:20.376274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:20.376274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:51:20.376274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:20.376274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:20.376274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:20.376274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:20.376274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:20.376274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:20.376274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:20.376274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:20.376274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:20.376274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:20.376274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:20.376274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:20.376274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:20.377789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:20.377789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:20.377789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:51:20.377789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:51:20.377789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:51:20.377789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:51:20.377789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:51:20.377789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:51:20.377789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 17:51:20.377789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:20.379392 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 373/763] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:51:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:20.379889 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d4870-cd37-4efa-8363-0a6b0c855307 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:20.407918 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc63fc-0adf-48dc-bfe3-015d43f3cf29 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:20.452861 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522e0a4-027c-47e0-91c5-b827eb0efdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:20.453566 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522e0a4-027c-47e0-91c5-b827eb0efdf3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:20.454860 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522e0a4-027c-47e0-91c5-b827eb0efdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:20.455142 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522e0a4-027c-47e0-91c5-b827eb0efdf3 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:20.455942 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522e0a4-027c-47e0-91c5-b827eb0efdf3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:20.456253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522e0a4-027c-47e0-91c5-b827eb0efdf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:20.457757 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522e0a4-027c-47e0-91c5-b827eb0efdf3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:20.469875 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 391/764] 10.222.0.76 () {56 vars in 1103 bytes} [Thu May 26 17:51:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:20.470561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522e0a4-027c-47e0-91c5-b827eb0efdf3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:20.516616 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d4870-cd37-4efa-8363-0a6b0c855307 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:20.545229 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:20.562089 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a461a-145b-475d-b5f8-5b3ee8236bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:20.563166 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a461a-145b-475d-b5f8-5b3ee8236bfc None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:20.563535 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a461a-145b-475d-b5f8-5b3ee8236bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:20.563991 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a461a-145b-475d-b5f8-5b3ee8236bfc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:20.565629 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a461a-145b-475d-b5f8-5b3ee8236bfc None admin] RBAC: Adding query filter params (interface=public, service_id=8848b7ed73d04c09a96d489b10d317cb, region_id=RegionOne) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:20.566298 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a461a-145b-475d-b5f8-5b3ee8236bfc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:20.568100 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a461a-145b-475d-b5f8-5b3ee8236bfc None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:20.578674 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 374/765] 10.222.0.76 () {56 vars in 1251 bytes} [Thu May 26 17:51:20 2022] GET /identity/v3/endpoints?service_id=8848b7ed73d04c09a96d489b10d317cb&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:20.579445 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a461a-145b-475d-b5f8-5b3ee8236bfc None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:20.612073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522e0a4-027c-47e0-91c5-b827eb0efdf3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:20.618796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:20.661343 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffedafdb-a235-4543-9a17-5fe81850f030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:20.661939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffedafdb-a235-4543-9a17-5fe81850f030 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:20.662252 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffedafdb-a235-4543-9a17-5fe81850f030 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:20.662530 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffedafdb-a235-4543-9a17-5fe81850f030 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:20.663277 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffedafdb-a235-4543-9a17-5fe81850f030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:20.663577 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffedafdb-a235-4543-9a17-5fe81850f030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:20.664961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffedafdb-a235-4543-9a17-5fe81850f030 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:20.678001 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 392/766] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:20.679462 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de3db7-93f4-43d0-9e7c-5c95d541166c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:20.679751 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de3db7-93f4-43d0-9e7c-5c95d541166c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:20.680070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de3db7-93f4-43d0-9e7c-5c95d541166c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:20.681569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 393/767] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:20.682039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de3db7-93f4-43d0-9e7c-5c95d541166c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:20.727490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a461a-145b-475d-b5f8-5b3ee8236bfc None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:20.774303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4668e-b9a6-4b3d-a50b-8d1d4427162a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:20.775295 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4668e-b9a6-4b3d-a50b-8d1d4427162a None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:20.775749 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4668e-b9a6-4b3d-a50b-8d1d4427162a None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:20.776170 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4668e-b9a6-4b3d-a50b-8d1d4427162a None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:20.777474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4668e-b9a6-4b3d-a50b-8d1d4427162a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:20.778025 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4668e-b9a6-4b3d-a50b-8d1d4427162a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:20.780260 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4668e-b9a6-4b3d-a50b-8d1d4427162a None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:20.793875 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de3db7-93f4-43d0-9e7c-5c95d541166c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:20.808677 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 375/768] 10.222.0.76 () {56 vars in 1135 bytes} [Thu May 26 17:51:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:20.809690 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4668e-b9a6-4b3d-a50b-8d1d4427162a None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:20.817655 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:20.847199 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07e72b-0962-4385-9bbe-2f373d2a1ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:20.848846 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07e72b-0962-4385-9bbe-2f373d2a1ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:20.849372 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07e72b-0962-4385-9bbe-2f373d2a1ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:20.849793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07e72b-0962-4385-9bbe-2f373d2a1ed9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:20.871359 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07e72b-0962-4385-9bbe-2f373d2a1ed9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:20.872896 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07e72b-0962-4385-9bbe-2f373d2a1ed9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:20.892535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-9b07e72b-0962-4385-9bbe-2f373d2a1ed9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:20.892535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:20.892535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:20.892535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:20.892535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:20.892535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:20.892535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:20.892535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:20.892535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:20.892535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:20.892535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:20.892535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:20.892535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:20.892535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:20.892535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:20.894641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:20.894641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:20.894641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:20.894641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:20.894641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:20.894641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:20.894641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:20.894641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:20.894641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:20.894641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:20.894641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:20.894641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:20.894641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:20.894641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:20.894641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:20.894641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:20.894641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:20.895680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:20.895680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:20.895680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:20.895680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:20.895680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 394/769] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:20.895680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07e72b-0962-4385-9bbe-2f373d2a1ed9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:20.928587 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:20.946949 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4668e-b9a6-4b3d-a50b-8d1d4427162a None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:20.990448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d51a2c-a988-4922-aea8-eff7d132b7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:20.991144 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d51a2c-a988-4922-aea8-eff7d132b7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:20.991533 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d51a2c-a988-4922-aea8-eff7d132b7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:20.991687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d51a2c-a988-4922-aea8-eff7d132b7e5 None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:20.992374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d51a2c-a988-4922-aea8-eff7d132b7e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:20.993767 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d51a2c-a988-4922-aea8-eff7d132b7e5 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:20.999531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 376/770] 10.222.0.76 () {56 vars in 1087 bytes} [Thu May 26 17:51:20 2022] GET /identity/v3/services => generated 772 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:21.000928 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8438157-a7d4-4c0c-a5e4-659f3403f0bc None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:21.001227 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8438157-a7d4-4c0c-a5e4-659f3403f0bc None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:21.001486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8438157-a7d4-4c0c-a5e4-659f3403f0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:21.025621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-f8438157-a7d4-4c0c-a5e4-659f3403f0bc None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:21.027209 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f8438157-a7d4-4c0c-a5e4-659f3403f0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:21.028237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07e72b-0962-4385-9bbe-2f373d2a1ed9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:21.078225 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547d1f5-d03d-4952-9bec-6e12b721fb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:21.079161 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547d1f5-d03d-4952-9bec-6e12b721fb3c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:21.079516 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547d1f5-d03d-4952-9bec-6e12b721fb3c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:21.079890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547d1f5-d03d-4952-9bec-6e12b721fb3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:21.088523 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547d1f5-d03d-4952-9bec-6e12b721fb3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:21.090235 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547d1f5-d03d-4952-9bec-6e12b721fb3c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:21.115330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-5547d1f5-d03d-4952-9bec-6e12b721fb3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:21.115330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:21.115330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:21.115330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:21.115330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:21.115330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:21.115330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:21.115330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:21.115330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:21.115330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:21.115330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:21.115330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:21.115330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:51:21.115330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:51:21.115330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:51:21.116522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:51:21.116522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:21.116522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:21.116522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:21.116522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:21.116522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:21.116522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:21.116522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:21.116522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:21.116522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:21.116522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:21.116522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:21.116522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:21.116522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:21.116522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:21.116522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:21.116522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:21.117570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:51:21.117570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:51:21.117570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:51:21.117570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:51:21.117570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:51:21.117570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:51:21.117570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:21.117570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:21.121711 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 395/771] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:21.122860 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547d1f5-d03d-4952-9bec-6e12b721fb3c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:21.125217 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 377/772] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:21.139240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8438157-a7d4-4c0c-a5e4-659f3403f0bc None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:21.176378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:21.265319 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:21.294287 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8438157-a7d4-4c0c-a5e4-659f3403f0bc None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:21.302244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547d1f5-d03d-4952-9bec-6e12b721fb3c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:21.353546 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc275a0-59d8-490b-b261-7f6c8ec62d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:21.354751 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc275a0-59d8-490b-b261-7f6c8ec62d34 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:21.355282 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc275a0-59d8-490b-b261-7f6c8ec62d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:21.355520 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb44587d-f260-4980-b802-abc60dc40059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:21.355805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc275a0-59d8-490b-b261-7f6c8ec62d34 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:21.356497 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44587d-f260-4980-b802-abc60dc40059 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:21.356935 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44587d-f260-4980-b802-abc60dc40059 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:21.357222 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc275a0-59d8-490b-b261-7f6c8ec62d34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:21.357318 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44587d-f260-4980-b802-abc60dc40059 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:21.357797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc275a0-59d8-490b-b261-7f6c8ec62d34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:21.360495 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc275a0-59d8-490b-b261-7f6c8ec62d34 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:21.387687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 378/773] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:21.387687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc275a0-59d8-490b-b261-7f6c8ec62d34 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:21.492039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb44587d-f260-4980-b802-abc60dc40059 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:21.494049 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb44587d-f260-4980-b802-abc60dc40059 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:21.530340 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:21.537721 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc275a0-59d8-490b-b261-7f6c8ec62d34 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:21.576979 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257d4cd-d6ad-4c0f-ab36-2314f5313e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:21.577658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257d4cd-d6ad-4c0f-ab36-2314f5313e8a None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:21.577945 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257d4cd-d6ad-4c0f-ab36-2314f5313e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:21.578253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257d4cd-d6ad-4c0f-ab36-2314f5313e8a None admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:21.579055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257d4cd-d6ad-4c0f-ab36-2314f5313e8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:21.579495 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257d4cd-d6ad-4c0f-ab36-2314f5313e8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:21.581484 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257d4cd-d6ad-4c0f-ab36-2314f5313e8a None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:21.592212 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:21.597448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 379/774] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:51:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:21.598728 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257d4cd-d6ad-4c0f-ab36-2314f5313e8a None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:21.671116 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 396/775] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:51:21 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:51:21.673092 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb44587d-f260-4980-b802-abc60dc40059 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:21.745601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257d4cd-d6ad-4c0f-ab36-2314f5313e8a None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:21.788837 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f2684-9346-47b1-bf41-c8211a917ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:21.789743 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f2684-9346-47b1-bf41-c8211a917ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:21.790028 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f2684-9346-47b1-bf41-c8211a917ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:21.790358 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f2684-9346-47b1-bf41-c8211a917ed6 None admin] PATH_INFO: `/v3/services/network` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:21.791089 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f2684-9346-47b1-bf41-c8211a917ed6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:21.792609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f2684-9346-47b1-bf41-c8211a917ed6 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:21.809675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-a96f2684-9346-47b1-bf41-c8211a917ed6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 17:51:21.809675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:21.809675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:21.809675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:21.809675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:21.809675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:21.809675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:21.809675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:21.809675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 17:51:21.809675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:21.809675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:21.809675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:21.809675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:21.809675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:21.809675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:21.809675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:21.809675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:21.810810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:21.810810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:21.810810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:21.810810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:21.810810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:21.810810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:21.810810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:21.810810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:21.810810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:21.810810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:21.810810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:21.810810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:21.810810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:21.810810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:21.810810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:21.810810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:21.811678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:21.811678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:21.811678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:21.811678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:21.811678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:21.811678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:21.811678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:21.811678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:21.811678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:21.811678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:21.811678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:21.811678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:21.811678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 17:51:21.811678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:21.811678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 380/776] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:21 2022] GET /identity/v3/services/network => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:21.812510 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f2684-9346-47b1-bf41-c8211a917ed6 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:21.848084 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb44587d-f260-4980-b802-abc60dc40059 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:21.882642 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:21.896850 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044de09f-a0bd-42cb-a2a2-c5eb96ed2c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:21.897689 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044de09f-a0bd-42cb-a2a2-c5eb96ed2c66 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:21.897970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044de09f-a0bd-42cb-a2a2-c5eb96ed2c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:21.898348 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044de09f-a0bd-42cb-a2a2-c5eb96ed2c66 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:21.907523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:21.915473 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044de09f-a0bd-42cb-a2a2-c5eb96ed2c66 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:21.916953 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044de09f-a0bd-42cb-a2a2-c5eb96ed2c66 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:21.936284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-044de09f-a0bd-42cb-a2a2-c5eb96ed2c66 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:51:21.936284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:21.936284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:21.936284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:21.936284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:21.936284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:21.936284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:21.936284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:21.936284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:21.936284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:21.936284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:21.936284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:21.936284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:51:21.936284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:51:21.936284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:51:21.936284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:51:21.937299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:21.937299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:21.937299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:51:21.937299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:21.937299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:51:21.937299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:21.937299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:21.937299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:21.937299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:21.937299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:21.937299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:21.937299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:21.937299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:21.937299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:21.937299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:21.937299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:21.937299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:21.937299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:21.938269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:21.938269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:21.938269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:51:21.938269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:51:21.938269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:51:21.938269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:51:21.938269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:51:21.938269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:51:21.938269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:51:21.938269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:21.938269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 397/777] 10.222.0.76 () {56 vars in 1117 bytes} [Thu May 26 17:51:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:21.938975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044de09f-a0bd-42cb-a2a2-c5eb96ed2c66 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:21.974964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f2684-9346-47b1-bf41-c8211a917ed6 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:22.017578 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b07b5a-ab4c-41b2-a874-0285b6315c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:22.019347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b07b5a-ab4c-41b2-a874-0285b6315c27 None admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:22.019699 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b07b5a-ab4c-41b2-a874-0285b6315c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:22.019977 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b07b5a-ab4c-41b2-a874-0285b6315c27 None admin] PATH_INFO: `/v3/projects/ca1d09f491d944f99425a641561b03c3/users/1899c317083d40f5b490cd9eb63fe521/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:22.022385 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b07b5a-ab4c-41b2-a874-0285b6315c27 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1d09f491d944f99425a641561b03c3, user_id=1899c317083d40f5b490cd9eb63fe521, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:22.023940 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b07b5a-ab4c-41b2-a874-0285b6315c27 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:22.050331 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 381/778] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:51:21 2022] PUT /identity/v3/projects/ca1d09f491d944f99425a641561b03c3/users/1899c317083d40f5b490cd9eb63fe521/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:51:22.050331 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b07b5a-ab4c-41b2-a874-0285b6315c27 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:22.125524 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044de09f-a0bd-42cb-a2a2-c5eb96ed2c66 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:22.206771 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6950ce-235f-4b90-a52a-2c116284b2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:22.208210 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6950ce-235f-4b90-a52a-2c116284b2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:22.208701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6950ce-235f-4b90-a52a-2c116284b2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:22.209301 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6950ce-235f-4b90-a52a-2c116284b2b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:22.231976 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6950ce-235f-4b90-a52a-2c116284b2b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:22.234522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6950ce-235f-4b90-a52a-2c116284b2b1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:22.258497 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b07b5a-ab4c-41b2-a874-0285b6315c27 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:22.272103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-7f6950ce-235f-4b90-a52a-2c116284b2b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:22.272103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:22.272103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:22.272103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:22.272103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:22.272103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:22.272103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:22.272103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:22.272103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:22.272103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:22.272103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:22.272103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:22.272103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:22.272103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:22.272103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:22.273909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:22.273909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:22.273909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:22.273909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:22.273909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:22.273909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:22.273909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:22.273909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:22.273909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:22.273909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:22.273909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:22.273909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:22.273909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:22.273909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:22.273909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:22.273909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:22.273909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:22.276043 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:22.276043 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:22.276043 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:22.276043 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:22.276043 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 398/779] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:22.276760 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6950ce-235f-4b90-a52a-2c116284b2b1 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:22.315196 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:22.347102 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a04133-a49e-4d97-9bdc-8135033f7a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:22.348893 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a04133-a49e-4d97-9bdc-8135033f7a23 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:22.349835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a04133-a49e-4d97-9bdc-8135033f7a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:22.350629 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a04133-a49e-4d97-9bdc-8135033f7a23 None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:22.352044 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a04133-a49e-4d97-9bdc-8135033f7a23 None admin] RBAC: Adding query filter params (name=network) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:22.353102 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a04133-a49e-4d97-9bdc-8135033f7a23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:22.356910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a04133-a49e-4d97-9bdc-8135033f7a23 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:22.358542 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:22.369520 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 382/780] 10.222.0.76 () {56 vars in 1112 bytes} [Thu May 26 17:51:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:22.370519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a04133-a49e-4d97-9bdc-8135033f7a23 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:22.467790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6950ce-235f-4b90-a52a-2c116284b2b1 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:22.496230 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a04133-a49e-4d97-9bdc-8135033f7a23 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:22.522525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6041241-5afa-44c0-a299-c210c0395cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:22.523285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6041241-5afa-44c0-a299-c210c0395cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:22.523575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6041241-5afa-44c0-a299-c210c0395cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:22.523908 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6041241-5afa-44c0-a299-c210c0395cd2 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:22.524844 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6041241-5afa-44c0-a299-c210c0395cd2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:22.525230 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6041241-5afa-44c0-a299-c210c0395cd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:22.527875 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6041241-5afa-44c0-a299-c210c0395cd2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:22.553728 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee962cf-6143-40ef-b2fd-2361c751b5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:22.555118 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee962cf-6143-40ef-b2fd-2361c751b5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:22.555765 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee962cf-6143-40ef-b2fd-2361c751b5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:22.556383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee962cf-6143-40ef-b2fd-2361c751b5f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:22.556591 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 399/781] 10.222.0.76 () {56 vars in 1149 bytes} [Thu May 26 17:51:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:22.558019 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee962cf-6143-40ef-b2fd-2361c751b5f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:22.558716 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee962cf-6143-40ef-b2fd-2361c751b5f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:22.561408 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee962cf-6143-40ef-b2fd-2361c751b5f1 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:22.562792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6041241-5afa-44c0-a299-c210c0395cd2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:22.586509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 383/782] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:22.588116 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee962cf-6143-40ef-b2fd-2361c751b5f1 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:22.665656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:22.732514 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:22.740215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee962cf-6143-40ef-b2fd-2361c751b5f1 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:22.746912 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6041241-5afa-44c0-a299-c210c0395cd2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:22.781195 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2853e1-8425-46e5-8fa3-b5f4e9dc86a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:22.781968 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2853e1-8425-46e5-8fa3-b5f4e9dc86a2 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:22.782316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2853e1-8425-46e5-8fa3-b5f4e9dc86a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:22.782655 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2853e1-8425-46e5-8fa3-b5f4e9dc86a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:22.801733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2853e1-8425-46e5-8fa3-b5f4e9dc86a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:22.804457 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2853e1-8425-46e5-8fa3-b5f4e9dc86a2 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:22.827331 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9bee7e-07e4-4c07-96af-6d7a8cc2245d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:22.828998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9bee7e-07e4-4c07-96af-6d7a8cc2245d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:22.829758 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9bee7e-07e4-4c07-96af-6d7a8cc2245d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:22.831054 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9bee7e-07e4-4c07-96af-6d7a8cc2245d None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:22.832691 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9bee7e-07e4-4c07-96af-6d7a8cc2245d None admin] RBAC: Adding query filter params (type=network) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:22.833554 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9bee7e-07e4-4c07-96af-6d7a8cc2245d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:22.836497 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9bee7e-07e4-4c07-96af-6d7a8cc2245d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:22.838317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-0b2853e1-8425-46e5-8fa3-b5f4e9dc86a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:22.838317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:22.838317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:22.838317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:22.838317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:22.838317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:22.838317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:22.838317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:22.838317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:22.838317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:22.838317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:22.838317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:22.838317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:22.838317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:22.838317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:22.840055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:22.840055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:22.840055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:22.840055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:22.840055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:22.840055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:22.840055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:22.840055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:22.840055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:22.840055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:22.840055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:22.840055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:22.840055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:22.840055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:22.840055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:22.840055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:22.840055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:22.842613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:22.842613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:22.842613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:22.842613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:22.846561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 384/783] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:22.848453 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2853e1-8425-46e5-8fa3-b5f4e9dc86a2 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:22.860185 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 400/784] 10.222.0.76 () {56 vars in 1112 bytes} [Thu May 26 17:51:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:22.860185 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9bee7e-07e4-4c07-96af-6d7a8cc2245d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:23.040581 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:23.048820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2853e1-8425-46e5-8fa3-b5f4e9dc86a2 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:23.065987 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9bee7e-07e4-4c07-96af-6d7a8cc2245d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:23.122529 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941dbdf1-c123-4079-8b09-7f58e3ff1f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:23.123019 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346c251-90ac-42c0-a0ff-5386b1a1e2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:23.123846 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941dbdf1-c123-4079-8b09-7f58e3ff1f64 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:23.124404 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941dbdf1-c123-4079-8b09-7f58e3ff1f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:23.124547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346c251-90ac-42c0-a0ff-5386b1a1e2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:23.124933 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346c251-90ac-42c0-a0ff-5386b1a1e2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:23.125391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941dbdf1-c123-4079-8b09-7f58e3ff1f64 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:23.125558 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346c251-90ac-42c0-a0ff-5386b1a1e2f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:23.127442 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941dbdf1-c123-4079-8b09-7f58e3ff1f64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:23.128450 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941dbdf1-c123-4079-8b09-7f58e3ff1f64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:23.131322 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941dbdf1-c123-4079-8b09-7f58e3ff1f64 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:23.144290 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346c251-90ac-42c0-a0ff-5386b1a1e2f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:23.147637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346c251-90ac-42c0-a0ff-5386b1a1e2f0 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:23.157074 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 401/785] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:23.159162 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941dbdf1-c123-4079-8b09-7f58e3ff1f64 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:23.173887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-c346c251-90ac-42c0-a0ff-5386b1a1e2f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:23.173887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:23.173887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:23.173887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:23.173887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:23.173887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:23.173887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:23.173887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:23.173887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:23.173887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:23.173887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:23.173887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:23.173887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:51:23.173887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:51:23.173887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:51:23.175939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:51:23.175939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:23.175939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:23.175939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:23.175939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:23.175939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:23.175939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:23.175939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:23.175939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:23.175939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:23.175939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:23.175939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:23.175939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:23.175939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:23.175939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:23.175939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:23.175939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:23.177787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:51:23.177787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:51:23.177787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:51:23.177787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:51:23.177787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:51:23.177787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:51:23.177787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:23.177787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:23.177787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 385/786] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:23.177787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346c251-90ac-42c0-a0ff-5386b1a1e2f0 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:23.209277 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:23.318976 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941dbdf1-c123-4079-8b09-7f58e3ff1f64 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:23.321134 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346c251-90ac-42c0-a0ff-5386b1a1e2f0 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:23.366152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bef909-1894-4b6f-b06b-258973abeaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:23.367171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d88f0-bbe1-4d39-845a-5bd0b1092334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:23.367560 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bef909-1894-4b6f-b06b-258973abeaf3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:23.367983 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d88f0-bbe1-4d39-845a-5bd0b1092334 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:23.368291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d88f0-bbe1-4d39-845a-5bd0b1092334 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:23.368534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d88f0-bbe1-4d39-845a-5bd0b1092334 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:23.368752 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bef909-1894-4b6f-b06b-258973abeaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:23.369237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bef909-1894-4b6f-b06b-258973abeaf3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:23.370304 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bef909-1894-4b6f-b06b-258973abeaf3 None admin] RBAC: Adding query filter params (interface=public, service_id=4378681bf6c8489fa099ecf5039cad29, region_id=RegionOne) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:23.370989 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bef909-1894-4b6f-b06b-258973abeaf3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:23.372825 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bef909-1894-4b6f-b06b-258973abeaf3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:23.377299 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d88f0-bbe1-4d39-845a-5bd0b1092334 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:23.379702 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d88f0-bbe1-4d39-845a-5bd0b1092334 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:23.388017 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 402/787] 10.222.0.76 () {56 vars in 1251 bytes} [Thu May 26 17:51:23 2022] GET /identity/v3/endpoints?service_id=4378681bf6c8489fa099ecf5039cad29&interface=public®ion_id=RegionOne => generated 194 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:23.389441 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bef909-1894-4b6f-b06b-258973abeaf3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:23.399456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-1f5d88f0-bbe1-4d39-845a-5bd0b1092334 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:23.399456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:23.399456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:23.399456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:23.399456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:23.399456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:23.399456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:23.399456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:23.399456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:23.399456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:23.399456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:23.399456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:23.399456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:51:23.399456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:51:23.399456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:51:23.400415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:51:23.400415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:23.400415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:23.400415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:23.400415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:23.400415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:23.400415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:23.400415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:23.400415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:23.400415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:23.400415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:23.400415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:23.400415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:23.400415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:23.400415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:23.400415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:23.400415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:23.401761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:51:23.401761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:51:23.401761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:51:23.401761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:51:23.401761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:51:23.401761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:51:23.401761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:23.401761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:23.406122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 386/788] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:23.407316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d88f0-bbe1-4d39-845a-5bd0b1092334 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:23.439787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:23.509085 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:23.529614 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bef909-1894-4b6f-b06b-258973abeaf3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:23.560333 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d88f0-bbe1-4d39-845a-5bd0b1092334 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:23.600066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090d443-4f0e-44bb-b715-301d12392db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:23.600315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de137f0c-62e5-4c01-a817-6abb13d1f159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:23.600797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de137f0c-62e5-4c01-a817-6abb13d1f159 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:23.600967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090d443-4f0e-44bb-b715-301d12392db2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:23.601082 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de137f0c-62e5-4c01-a817-6abb13d1f159 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:23.601372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de137f0c-62e5-4c01-a817-6abb13d1f159 None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:23.601476 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090d443-4f0e-44bb-b715-301d12392db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:23.601863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090d443-4f0e-44bb-b715-301d12392db2 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:23.602036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de137f0c-62e5-4c01-a817-6abb13d1f159 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:23.602891 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090d443-4f0e-44bb-b715-301d12392db2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:23.603465 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de137f0c-62e5-4c01-a817-6abb13d1f159 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:23.603563 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090d443-4f0e-44bb-b715-301d12392db2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:23.605776 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090d443-4f0e-44bb-b715-301d12392db2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:23.610136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 387/789] 10.222.0.76 () {56 vars in 1087 bytes} [Thu May 26 17:51:23 2022] GET /identity/v3/services => generated 772 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:23.610467 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de137f0c-62e5-4c01-a817-6abb13d1f159 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:23.636563 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 403/790] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:51:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:23.639114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c56f92-763d-47ce-ac1a-f9305b353f77 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:23.639570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c56f92-763d-47ce-ac1a-f9305b353f77 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:23.640010 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c56f92-763d-47ce-ac1a-f9305b353f77 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:23.643784 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 404/791] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:23.645325 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c56f92-763d-47ce-ac1a-f9305b353f77 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:23.726438 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:23.745917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de137f0c-62e5-4c01-a817-6abb13d1f159 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:23.787651 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca1277-bf36-4fb6-983d-894cbb4137d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:23.788502 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca1277-bf36-4fb6-983d-894cbb4137d0 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:23.788944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca1277-bf36-4fb6-983d-894cbb4137d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:23.789328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca1277-bf36-4fb6-983d-894cbb4137d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:23.790291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca1277-bf36-4fb6-983d-894cbb4137d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:23.790714 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca1277-bf36-4fb6-983d-894cbb4137d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:23.792307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca1277-bf36-4fb6-983d-894cbb4137d0 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:23.808507 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 388/792] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:51:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:23.810210 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65330e98-c5d5-4350-ba75-09c6c31dd92b None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:23.810668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65330e98-c5d5-4350-ba75-09c6c31dd92b None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:23.811057 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65330e98-c5d5-4350-ba75-09c6c31dd92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:23.830561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-65330e98-c5d5-4350-ba75-09c6c31dd92b None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:23.832055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-65330e98-c5d5-4350-ba75-09c6c31dd92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:23.855287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c56f92-763d-47ce-ac1a-f9305b353f77 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:23.919029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1531801-9edb-45ec-b5c4-07528062c719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:23.920211 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1531801-9edb-45ec-b5c4-07528062c719 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:23.920734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1531801-9edb-45ec-b5c4-07528062c719 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:23.921357 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1531801-9edb-45ec-b5c4-07528062c719 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:23.922986 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1531801-9edb-45ec-b5c4-07528062c719 None admin] RBAC: Adding query filter params (role.id=4fa8f06085c741a5b3cf64e63cc0754f, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=9ca774e864ca44cfb187e38bc8a8f6ea) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:23.923541 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1531801-9edb-45ec-b5c4-07528062c719 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:23.926549 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1531801-9edb-45ec-b5c4-07528062c719 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:23.929570 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 389/793] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:23.934919 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 405/794] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:51:23 2022] GET /identity/v3/role_assignments?user.id=9ca774e864ca44cfb187e38bc8a8f6ea&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=4fa8f06085c741a5b3cf64e63cc0754f => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:23.935970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1531801-9edb-45ec-b5c4-07528062c719 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:23.952113 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:23.957841 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65330e98-c5d5-4350-ba75-09c6c31dd92b None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:24.070989 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1531801-9edb-45ec-b5c4-07528062c719 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:24.117325 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d00111-9851-415c-959a-56f405db69b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:24.118500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d00111-9851-415c-959a-56f405db69b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:24.118901 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d00111-9851-415c-959a-56f405db69b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:24.119320 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d00111-9851-415c-959a-56f405db69b2 None admin] PATH_INFO: `/v3/projects/ca1d09f491d944f99425a641561b03c3/users/3e06d1b0e1a9472db764f4be7e42fdd9/roles/e776303775a2423aa0c62c161ecb5a7e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:24.151313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d00111-9851-415c-959a-56f405db69b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1d09f491d944f99425a641561b03c3, user_id=3e06d1b0e1a9472db764f4be7e42fdd9, role_id=e776303775a2423aa0c62c161ecb5a7e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:24.153053 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d00111-9851-415c-959a-56f405db69b2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:24.182850 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:24.183041 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 406/795] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:51:23 2022] PUT /identity/v3/projects/ca1d09f491d944f99425a641561b03c3/users/3e06d1b0e1a9472db764f4be7e42fdd9/roles/e776303775a2423aa0c62c161ecb5a7e => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:51:24.214722 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65330e98-c5d5-4350-ba75-09c6c31dd92b None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:24.273536 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8e623-cb91-4451-a376-592bb44db1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:24.274412 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8e623-cb91-4451-a376-592bb44db1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:24.274699 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8e623-cb91-4451-a376-592bb44db1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:24.274977 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8e623-cb91-4451-a376-592bb44db1a5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:24.275663 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8e623-cb91-4451-a376-592bb44db1a5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:24.277101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8e623-cb91-4451-a376-592bb44db1a5 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:24.292306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-5bd8e623-cb91-4451-a376-592bb44db1a5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 17:51:24.292306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:24.292306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:24.292306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:24.292306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:24.292306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:24.292306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:24.292306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:24.292306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 17:51:24.292306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:24.292306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:24.292306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:24.292306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:24.292306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:24.292306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:24.292306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:24.293321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:24.293321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:24.293321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:24.293321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:24.293321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:24.293321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:24.293321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:24.293321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:24.293321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:24.293321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:24.293321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:24.293321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:24.293321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:24.293321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:24.293321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:24.293321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:24.294649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:24.294649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:24.294649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:24.294649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:24.294649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:24.294649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:24.294649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:24.294649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:24.294649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:24.294649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:24.294649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:24.294649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:24.294649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:24.294649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 17:51:24.294649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:24.294649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 390/796] 10.222.0.76 () {56 vars in 1129 bytes} [Thu May 26 17:51:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:24.303511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d00111-9851-415c-959a-56f405db69b2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:24.447672 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:24.527607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d00111-9851-415c-959a-56f405db69b2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:24.588965 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff988a-6f95-45a1-bff1-c7cca670bb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:24.590029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff988a-6f95-45a1-bff1-c7cca670bb50 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:24.590452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff988a-6f95-45a1-bff1-c7cca670bb50 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:24.590835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff988a-6f95-45a1-bff1-c7cca670bb50 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:24.592070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ff988a-6f95-45a1-bff1-c7cca670bb50 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:24.592599 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ff988a-6f95-45a1-bff1-c7cca670bb50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:24.594809 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ff988a-6f95-45a1-bff1-c7cca670bb50 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:24.603186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 407/797] 10.222.0.76 () {56 vars in 1124 bytes} [Thu May 26 17:51:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:24.609365 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8e623-cb91-4451-a376-592bb44db1a5 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:24.797972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8e623-cb91-4451-a376-592bb44db1a5 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:24.849543 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2dfb6c-149e-42c8-aaed-711fb7594b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:24.850285 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2dfb6c-149e-42c8-aaed-711fb7594b76 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:24.850615 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2dfb6c-149e-42c8-aaed-711fb7594b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:24.850893 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2dfb6c-149e-42c8-aaed-711fb7594b76 None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:24.851666 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2dfb6c-149e-42c8-aaed-711fb7594b76 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:24.852013 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2dfb6c-149e-42c8-aaed-711fb7594b76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:24.853531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2dfb6c-149e-42c8-aaed-711fb7594b76 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:24.859557 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 391/798] 10.222.0.76 () {56 vars in 1124 bytes} [Thu May 26 17:51:24 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:24.866433 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff988a-6f95-45a1-bff1-c7cca670bb50 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:24.898516 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c7440-f1bb-4ca4-83b9-e8778dbf46d0 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:24.898847 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c7440-f1bb-4ca4-83b9-e8778dbf46d0 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:24.899109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c7440-f1bb-4ca4-83b9-e8778dbf46d0 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:24.901662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 392/799] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:51:24.909299 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7563963-4451-4105-8c6c-5eb2655ae3b0 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:24.909835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7563963-4451-4105-8c6c-5eb2655ae3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:24.910119 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7563963-4451-4105-8c6c-5eb2655ae3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:24.916659 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:24.933817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-d7563963-4451-4105-8c6c-5eb2655ae3b0 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:24.935565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-d7563963-4451-4105-8c6c-5eb2655ae3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:25.054571 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 393/800] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:51:25.068293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff988a-6f95-45a1-bff1-c7cca670bb50 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:25.094454 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:25.099152 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7563963-4451-4105-8c6c-5eb2655ae3b0 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:25.140102 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd634c-006f-497a-8f12-c39c8238609b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:25.140824 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd634c-006f-497a-8f12-c39c8238609b None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:25.141440 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd634c-006f-497a-8f12-c39c8238609b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:25.141440 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd634c-006f-497a-8f12-c39c8238609b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:25.142120 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd634c-006f-497a-8f12-c39c8238609b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:25.143686 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd634c-006f-497a-8f12-c39c8238609b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:25.185232 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 408/801] 10.222.0.76 () {60 vars in 1144 bytes} [Thu May 26 17:51:24 2022] POST /identity/v3/endpoints => generated 346 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:51:25.297789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7563963-4451-4105-8c6c-5eb2655ae3b0 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:25.342704 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:25.367871 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74de99d-d21f-40e9-b250-b5b0ba978a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:25.368766 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74de99d-d21f-40e9-b250-b5b0ba978a04 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:25.369141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74de99d-d21f-40e9-b250-b5b0ba978a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:25.369431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74de99d-d21f-40e9-b250-b5b0ba978a04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:25.385958 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74de99d-d21f-40e9-b250-b5b0ba978a04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:51:25.400051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-e74de99d-d21f-40e9-b250-b5b0ba978a04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:51:25.400051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:25.400051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:25.400051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:25.400051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:25.400051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:25.400051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:25.400051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:25.400051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:25.400051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:25.400051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:25.400051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:25.400051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:51:25.400051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:51:25.400051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:51:25.400051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:51:25.401780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:51:25.401780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:51:25.401780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:51:25.401780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:51:25.401780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:25.401780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:25.401780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:25.401780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:25.401780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:25.401780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:25.401780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:25.401780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:25.401780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:25.401780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:25.401780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:25.401780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:25.404138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:25.404138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:25.404138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:25.404138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:25.404138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:51:25.404138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:51:25.404138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:51:25.404138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:51:25.404138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:51:25.404138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:51:25.404138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:51:25.404138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:25.404138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 394/802] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:51:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:51:25.415031 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd634c-006f-497a-8f12-c39c8238609b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:25.595772 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd634c-006f-497a-8f12-c39c8238609b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:25.640465 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20822203-3be4-41da-91ff-7246e6e98d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:25.641184 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20822203-3be4-41da-91ff-7246e6e98d2a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:25.641534 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20822203-3be4-41da-91ff-7246e6e98d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:25.641816 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20822203-3be4-41da-91ff-7246e6e98d2a None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:25.642671 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20822203-3be4-41da-91ff-7246e6e98d2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:25.642986 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20822203-3be4-41da-91ff-7246e6e98d2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:25.644538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20822203-3be4-41da-91ff-7246e6e98d2a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:25.655234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 409/803] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:51:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:25.661383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74de99d-d21f-40e9-b250-b5b0ba978a04 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:25.837422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74de99d-d21f-40e9-b250-b5b0ba978a04 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:25.907598 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec162ecd-670a-4e70-8865-b63bfb4d4ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:25.908835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec162ecd-670a-4e70-8865-b63bfb4d4ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:25.909280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec162ecd-670a-4e70-8865-b63bfb4d4ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:25.909724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec162ecd-670a-4e70-8865-b63bfb4d4ad5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:25.937841 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec162ecd-670a-4e70-8865-b63bfb4d4ad5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:25.940994 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec162ecd-670a-4e70-8865-b63bfb4d4ad5 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:25.964305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-ec162ecd-670a-4e70-8865-b63bfb4d4ad5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:25.964305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:25.964305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:25.964305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:25.964305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:25.964305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:25.964305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:25.964305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:25.964305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:25.964305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:25.964305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:25.964305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:25.964305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:25.964305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:25.964305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:25.965274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:25.965274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:25.965274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:25.965274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:25.965274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:25.965274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:25.965274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:25.965274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:25.965274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:25.965274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:25.965274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:25.965274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:25.965274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:25.965274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:25.965274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:25.965274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:25.965274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:25.966266 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:25.966266 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:25.966266 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:25.966266 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:25.974214 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 395/804] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:25.974940 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20822203-3be4-41da-91ff-7246e6e98d2a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:26.039268 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:26.129542 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20822203-3be4-41da-91ff-7246e6e98d2a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:26.190564 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104865f-06c7-4401-befc-76f53ed22bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:26.191028 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104865f-06c7-4401-befc-76f53ed22bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:26.191317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104865f-06c7-4401-befc-76f53ed22bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:26.191648 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104865f-06c7-4401-befc-76f53ed22bc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:26.192394 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104865f-06c7-4401-befc-76f53ed22bc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:26.192705 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104865f-06c7-4401-befc-76f53ed22bc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:26.194098 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104865f-06c7-4401-befc-76f53ed22bc8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:26.211540 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 410/805] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:26.217270 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec162ecd-670a-4e70-8865-b63bfb4d4ad5 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:26.278784 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:26.362537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec162ecd-670a-4e70-8865-b63bfb4d4ad5 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:26.412263 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b75cd-b961-44a3-b8b9-4e0b2d5cb2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:26.413088 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b75cd-b961-44a3-b8b9-4e0b2d5cb2ca None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:26.413374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b75cd-b961-44a3-b8b9-4e0b2d5cb2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:26.413653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b75cd-b961-44a3-b8b9-4e0b2d5cb2ca None admin] PATH_INFO: `/v3/users/nova` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:26.422302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b75cd-b961-44a3-b8b9-4e0b2d5cb2ca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:26.423808 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b75cd-b961-44a3-b8b9-4e0b2d5cb2ca None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:26.439110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-3a7b75cd-b961-44a3-b8b9-4e0b2d5cb2ca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 17:51:26.439110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:26.439110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:26.439110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:26.439110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:26.439110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:26.439110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:26.439110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:26.439110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:26.439110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:26.439110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:26.439110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:26.439110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:51:26.439110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:51:26.439110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:51:26.439110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:51:26.440865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:26.440865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:26.440865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:51:26.440865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:26.440865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:51:26.440865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:26.440865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:26.440865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:26.440865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:26.440865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:26.440865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:26.440865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:26.440865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:26.440865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:26.440865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:26.440865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:26.440865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:26.440865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:26.441862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:26.441862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:26.441862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:51:26.441862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:51:26.441862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:51:26.441862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:51:26.441862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:51:26.441862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:51:26.441862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 17:51:26.441862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:26.441862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 396/806] 10.222.0.76 () {56 vars in 1093 bytes} [Thu May 26 17:51:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:26.449309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104865f-06c7-4401-befc-76f53ed22bc8 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:26.598037 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:26.606514 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104865f-06c7-4401-befc-76f53ed22bc8 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:26.655726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf079f1-a422-4203-822d-2120119edd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:26.656412 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf079f1-a422-4203-822d-2120119edd55 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:26.656697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf079f1-a422-4203-822d-2120119edd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:26.656973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf079f1-a422-4203-822d-2120119edd55 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:26.657818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf079f1-a422-4203-822d-2120119edd55 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:26.658129 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf079f1-a422-4203-822d-2120119edd55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:26.660185 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf079f1-a422-4203-822d-2120119edd55 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:26.685933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 411/807] 10.222.0.76 () {56 vars in 1133 bytes} [Thu May 26 17:51:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:26.693477 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b75cd-b961-44a3-b8b9-4e0b2d5cb2ca None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:26.725611 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03407df3-48b2-4f0f-b17a-d57f02d2f3d6 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:26.725958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03407df3-48b2-4f0f-b17a-d57f02d2f3d6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:26.726177 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03407df3-48b2-4f0f-b17a-d57f02d2f3d6 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:26.727851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 412/808] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:26.733441 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91346c52-214f-4506-b0ca-576edf517cf1 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:26.733726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91346c52-214f-4506-b0ca-576edf517cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:26.734003 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91346c52-214f-4506-b0ca-576edf517cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:26.764226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-91346c52-214f-4506-b0ca-576edf517cf1 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:26.765439 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-91346c52-214f-4506-b0ca-576edf517cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:26.884857 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:26.889587 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b75cd-b961-44a3-b8b9-4e0b2d5cb2ca None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:26.931366 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85062a-63e8-47d2-a1b8-73fb015bd58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:26.932034 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85062a-63e8-47d2-a1b8-73fb015bd58f None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:26.932311 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85062a-63e8-47d2-a1b8-73fb015bd58f None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:26.932587 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85062a-63e8-47d2-a1b8-73fb015bd58f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:26.936934 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 413/809] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:26.985872 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91346c52-214f-4506-b0ca-576edf517cf1 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:27.019922 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85062a-63e8-47d2-a1b8-73fb015bd58f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:27.021394 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85062a-63e8-47d2-a1b8-73fb015bd58f None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:27.136310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:27.142871 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 397/810] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:51:26 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:51:27.150870 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10da61-05eb-47c3-a50a-998548ca9424 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:27.150870 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10da61-05eb-47c3-a50a-998548ca9424 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:27.151024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10da61-05eb-47c3-a50a-998548ca9424 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:27.152734 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 398/811] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:27.155034 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10da61-05eb-47c3-a50a-998548ca9424 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:27.193947 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:27.211640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91346c52-214f-4506-b0ca-576edf517cf1 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:27.260974 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909704a-1c94-4ee8-8449-77a6e7694345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:27.261805 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909704a-1c94-4ee8-8449-77a6e7694345 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:27.262091 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909704a-1c94-4ee8-8449-77a6e7694345 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:27.262403 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909704a-1c94-4ee8-8449-77a6e7694345 None admin] PATH_INFO: `/v3/services/network` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:27.263393 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e909704a-1c94-4ee8-8449-77a6e7694345 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:27.265500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e909704a-1c94-4ee8-8449-77a6e7694345 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:27.278263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-e909704a-1c94-4ee8-8449-77a6e7694345 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 17:51:27.278263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:27.278263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:27.278263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:27.278263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:27.278263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:27.278263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:27.278263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:27.278263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 17:51:27.278263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:27.278263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:27.278263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:27.278263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:27.278263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:27.278263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:27.278263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:27.278263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:27.279296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:27.279296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:27.279296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:27.279296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:27.279296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:27.279296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:27.279296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:27.279296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:27.279296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:27.279296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:27.279296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:27.279296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:27.279296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:27.279296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:27.279296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:27.279296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:27.280137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:27.280137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:27.280137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:27.280137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:27.280137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:27.280137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:27.280137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:27.280137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:27.280137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:27.280137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:27.280137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:27.280137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:27.280137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 17:51:27.280137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:27.280137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 414/812] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:26 2022] GET /identity/v3/services/network => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:51:27.280954 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc58ab5-fb03-4ace-969b-eac9d4896351 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:27.280954 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc58ab5-fb03-4ace-969b-eac9d4896351 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:27.281443 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc58ab5-fb03-4ace-969b-eac9d4896351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:27.295024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10da61-05eb-47c3-a50a-998548ca9424 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:27.298564 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5dc58ab5-fb03-4ace-969b-eac9d4896351 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:27.300355 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5dc58ab5-fb03-4ace-969b-eac9d4896351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:27.335138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ee09b-e45a-4975-845b-8ae7c6e9f417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:27.335940 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ee09b-e45a-4975-845b-8ae7c6e9f417 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:27.336225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ee09b-e45a-4975-845b-8ae7c6e9f417 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:27.336502 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ee09b-e45a-4975-845b-8ae7c6e9f417 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:27.347611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ee09b-e45a-4975-845b-8ae7c6e9f417 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:27.349055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ee09b-e45a-4975-845b-8ae7c6e9f417 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:27.367917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-206ee09b-e45a-4975-845b-8ae7c6e9f417 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:27.367917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:27.367917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:27.367917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:27.367917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:27.367917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:27.367917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:27.367917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:27.367917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:27.367917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:27.367917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:27.367917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:27.367917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:27.367917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:27.367917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:27.368856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:27.368856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:27.368856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:27.368856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:27.368856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:27.368856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:27.368856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:27.368856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:27.368856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:27.368856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:27.368856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:27.368856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:27.368856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:27.368856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:27.368856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:27.368856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:27.368856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:27.369829 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:27.369829 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:27.369829 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:27.369829 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:27.369829 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 399/813] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:27.370663 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf37c71-d366-4b1e-8557-54ee4edbad4a None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:27.370849 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf37c71-d366-4b1e-8557-54ee4edbad4a None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:27.371126 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf37c71-d366-4b1e-8557-54ee4edbad4a None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:27.374309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 400/814] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:27.374309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf37c71-d366-4b1e-8557-54ee4edbad4a None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:27.386716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 415/815] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:27.386716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc58ab5-fb03-4ace-969b-eac9d4896351 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:27.446884 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:27.480717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:27.504810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf37c71-d366-4b1e-8557-54ee4edbad4a None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:27.507635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc58ab5-fb03-4ace-969b-eac9d4896351 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:27.547872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5bba3-f939-4cd8-b155-9a9bb7465f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:27.548461 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5bba3-f939-4cd8-b155-9a9bb7465f7b None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:27.548749 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5bba3-f939-4cd8-b155-9a9bb7465f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:27.549871 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5bba3-f939-4cd8-b155-9a9bb7465f7b None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:27.550775 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a5bba3-f939-4cd8-b155-9a9bb7465f7b None admin] RBAC: Adding query filter params (name=network) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:27.551099 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a5bba3-f939-4cd8-b155-9a9bb7465f7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:27.552531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a5bba3-f939-4cd8-b155-9a9bb7465f7b None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:27.553759 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2541ba-ffd1-42ff-a0ca-32e30a35dc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:27.554690 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2541ba-ffd1-42ff-a0ca-32e30a35dc0b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:27.555020 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2541ba-ffd1-42ff-a0ca-32e30a35dc0b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:27.555407 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2541ba-ffd1-42ff-a0ca-32e30a35dc0b None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:27.556176 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2541ba-ffd1-42ff-a0ca-32e30a35dc0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:27.556562 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2541ba-ffd1-42ff-a0ca-32e30a35dc0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:27.558121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2541ba-ffd1-42ff-a0ca-32e30a35dc0b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:27.564795 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 401/816] 10.222.0.76 () {56 vars in 1112 bytes} [Thu May 26 17:51:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:27.564795 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26687d3-1be2-4c28-9c42-86144c0ca069 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:27.564795 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26687d3-1be2-4c28-9c42-86144c0ca069 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:27.564795 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26687d3-1be2-4c28-9c42-86144c0ca069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:27.580204 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 416/817] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:27.580204 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2541ba-ffd1-42ff-a0ca-32e30a35dc0b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:27.580480 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-e26687d3-1be2-4c28-9c42-86144c0ca069 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:27.580480 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-e26687d3-1be2-4c28-9c42-86144c0ca069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:27.652293 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 402/818] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:27.652953 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26687d3-1be2-4c28-9c42-86144c0ca069 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:27.701123 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2541ba-ffd1-42ff-a0ca-32e30a35dc0b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:27.702574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:27.739546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18079fe3-4d94-40a5-af0e-2a389095490b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:27.740345 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18079fe3-4d94-40a5-af0e-2a389095490b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:27.740624 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18079fe3-4d94-40a5-af0e-2a389095490b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:27.740894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18079fe3-4d94-40a5-af0e-2a389095490b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:27.751579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18079fe3-4d94-40a5-af0e-2a389095490b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:51:27.761601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-18079fe3-4d94-40a5-af0e-2a389095490b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:51:27.761601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:27.761601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:27.761601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:27.761601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:27.761601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:27.761601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:27.761601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:27.761601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:27.761601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:27.761601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:27.761601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:27.761601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:51:27.761601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:51:27.761601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:51:27.761601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:51:27.762652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:51:27.762652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:51:27.762652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:51:27.762652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:51:27.762652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:27.762652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:27.762652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:27.762652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:27.762652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:27.762652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:27.762652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:27.762652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:27.762652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:27.762652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:27.762652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:27.762652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:27.768758 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:27.768758 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:27.768758 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:27.768758 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:27.768758 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:51:27.768758 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:51:27.768758 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:51:27.768758 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:51:27.768758 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:51:27.768758 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:51:27.768758 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:51:27.768758 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:27.769744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26687d3-1be2-4c28-9c42-86144c0ca069 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:27.770793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 417/819] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:51:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:27.770793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18079fe3-4d94-40a5-af0e-2a389095490b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:27.770793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:27.810904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d04b5-f9a2-4400-96cd-1bab279014ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:27.811498 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d04b5-f9a2-4400-96cd-1bab279014ba None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:27.811834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d04b5-f9a2-4400-96cd-1bab279014ba None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:27.812109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d04b5-f9a2-4400-96cd-1bab279014ba None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:27.812841 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d04b5-f9a2-4400-96cd-1bab279014ba None admin] RBAC: Adding query filter params (type=network) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:27.813149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d04b5-f9a2-4400-96cd-1bab279014ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:27.814581 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d04b5-f9a2-4400-96cd-1bab279014ba None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:27.825922 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 403/820] 10.222.0.76 () {56 vars in 1112 bytes} [Thu May 26 17:51:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:27.826907 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d04b5-f9a2-4400-96cd-1bab279014ba None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:27.876085 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18079fe3-4d94-40a5-af0e-2a389095490b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:27.915070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc47d34-1b0c-4175-a929-493d94bd73f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:27.915911 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc47d34-1b0c-4175-a929-493d94bd73f9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:27.916200 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc47d34-1b0c-4175-a929-493d94bd73f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:27.916481 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc47d34-1b0c-4175-a929-493d94bd73f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:27.923717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc47d34-1b0c-4175-a929-493d94bd73f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:27.925281 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc47d34-1b0c-4175-a929-493d94bd73f9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:27.939764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-3bc47d34-1b0c-4175-a929-493d94bd73f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:27.939764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:27.939764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:27.939764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:27.939764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:27.939764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:27.939764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:27.939764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:27.939764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:27.939764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:27.939764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:27.939764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:27.939764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:51:27.939764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:51:27.939764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:51:27.940676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:51:27.940676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:27.940676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:27.940676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:27.940676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:27.940676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:27.940676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:27.940676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:27.940676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:27.940676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:27.940676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:27.940676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:27.940676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:27.940676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:27.940676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:27.940676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:27.940676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:27.941573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:51:27.941573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:51:27.941573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:51:27.941573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:51:27.941573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:51:27.941573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:51:27.941573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:27.941573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:27.941573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 418/821] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:27.942589 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc47d34-1b0c-4175-a929-493d94bd73f9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:27.948633 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d04b5-f9a2-4400-96cd-1bab279014ba None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:27.961597 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:27.994508 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc5f3e-baf5-4323-a003-8afea4d75676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:27.995941 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc5f3e-baf5-4323-a003-8afea4d75676 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:27.996403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc5f3e-baf5-4323-a003-8afea4d75676 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:27.997040 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc5f3e-baf5-4323-a003-8afea4d75676 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:28.012134 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc5f3e-baf5-4323-a003-8afea4d75676 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:51:28.015513 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:28.021354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-e4fc5f3e-baf5-4323-a003-8afea4d75676 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:51:28.021354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:28.021354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:28.021354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:28.021354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:28.021354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:28.021354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:28.021354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:28.021354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:28.021354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:28.021354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:28.021354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:28.021354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:51:28.021354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:51:28.021354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:51:28.021354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:51:28.022556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:51:28.022556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:51:28.022556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:51:28.022556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:51:28.022556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:28.022556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:28.022556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:28.022556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:28.022556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:28.022556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:28.022556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:28.022556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:28.022556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:28.022556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:28.022556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:28.022556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:28.023415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:28.023415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:28.023415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:28.023415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:28.023415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:51:28.023415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:51:28.023415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:51:28.023415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:51:28.023415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:51:28.023415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:51:28.023415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:51:28.023415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:28.023415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 404/822] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:51:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:28.024595 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc5f3e-baf5-4323-a003-8afea4d75676 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:28.066309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc47d34-1b0c-4175-a929-493d94bd73f9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:28.106461 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821d3ca-4585-4ff7-bf96-a6d4e6d1997b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:28.107149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821d3ca-4585-4ff7-bf96-a6d4e6d1997b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:28.107429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821d3ca-4585-4ff7-bf96-a6d4e6d1997b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:28.107707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821d3ca-4585-4ff7-bf96-a6d4e6d1997b None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:28.108462 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821d3ca-4585-4ff7-bf96-a6d4e6d1997b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:28.108774 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821d3ca-4585-4ff7-bf96-a6d4e6d1997b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:28.110155 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821d3ca-4585-4ff7-bf96-a6d4e6d1997b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:28.120023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 419/823] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:51:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:28.120658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821d3ca-4585-4ff7-bf96-a6d4e6d1997b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:28.141832 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc5f3e-baf5-4323-a003-8afea4d75676 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:28.178286 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b293e94-9d2e-4c82-980b-a363d1695e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:28.178958 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b293e94-9d2e-4c82-980b-a363d1695e7d None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:28.179244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b293e94-9d2e-4c82-980b-a363d1695e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:28.179521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b293e94-9d2e-4c82-980b-a363d1695e7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:28.180271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b293e94-9d2e-4c82-980b-a363d1695e7d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:28.181646 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b293e94-9d2e-4c82-980b-a363d1695e7d None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:28.203136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 405/824] 10.222.0.76 () {60 vars in 1144 bytes} [Thu May 26 17:51:28 2022] POST /identity/v3/endpoints => generated 327 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:51:28.204106 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b293e94-9d2e-4c82-980b-a363d1695e7d None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:28.238604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:28.258430 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821d3ca-4585-4ff7-bf96-a6d4e6d1997b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:28.288100 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:28.310280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551e57b-337e-46ea-895f-8d6d9f00d1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:28.311496 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551e57b-337e-46ea-895f-8d6d9f00d1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:28.311838 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551e57b-337e-46ea-895f-8d6d9f00d1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:28.312083 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551e57b-337e-46ea-895f-8d6d9f00d1c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:28.312877 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551e57b-337e-46ea-895f-8d6d9f00d1c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:28.313193 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551e57b-337e-46ea-895f-8d6d9f00d1c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:28.315363 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551e57b-337e-46ea-895f-8d6d9f00d1c3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:28.334596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 420/825] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:51:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:28.334596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551e57b-337e-46ea-895f-8d6d9f00d1c3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:28.338868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b293e94-9d2e-4c82-980b-a363d1695e7d None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:28.380416 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5262076-9a18-45fd-9301-6a7b17bbe594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:28.381106 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5262076-9a18-45fd-9301-6a7b17bbe594 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:28.381388 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5262076-9a18-45fd-9301-6a7b17bbe594 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:28.381672 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5262076-9a18-45fd-9301-6a7b17bbe594 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:28.382471 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5262076-9a18-45fd-9301-6a7b17bbe594 None admin] RBAC: Adding query filter params (name=service) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:28.382780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5262076-9a18-45fd-9301-6a7b17bbe594 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:28.384176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5262076-9a18-45fd-9301-6a7b17bbe594 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:28.394171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 406/826] 10.222.0.76 () {56 vars in 1103 bytes} [Thu May 26 17:51:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:28.395673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9cf5e-6be0-4689-af9c-38558f2b798e None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:28.395974 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9cf5e-6be0-4689-af9c-38558f2b798e None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:28.396258 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9cf5e-6be0-4689-af9c-38558f2b798e None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:28.397579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 407/827] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:28.398520 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9cf5e-6be0-4689-af9c-38558f2b798e None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:28.465187 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551e57b-337e-46ea-895f-8d6d9f00d1c3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:28.505500 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:28.506031 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82586198-b0ef-4a70-afa8-da6c3d4e0f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:28.506994 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82586198-b0ef-4a70-afa8-da6c3d4e0f8c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:28.507394 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82586198-b0ef-4a70-afa8-da6c3d4e0f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:28.507840 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82586198-b0ef-4a70-afa8-da6c3d4e0f8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:28.521176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9cf5e-6be0-4689-af9c-38558f2b798e None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:28.522605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82586198-b0ef-4a70-afa8-da6c3d4e0f8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:28.524372 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82586198-b0ef-4a70-afa8-da6c3d4e0f8c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:28.546636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-82586198-b0ef-4a70-afa8-da6c3d4e0f8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:28.546636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:28.546636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:28.546636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:28.546636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:28.546636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:28.546636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:28.546636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:28.546636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:28.546636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:28.546636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:28.546636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:28.546636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:28.546636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:28.546636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:28.548470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:28.548470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:28.548470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:28.548470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:28.548470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:28.548470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:28.548470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:28.548470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:28.548470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:28.548470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:28.548470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:28.548470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:28.548470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:28.548470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:28.548470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:28.548470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:28.548470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:28.550949 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:28.550949 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:28.550949 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:28.550949 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:28.550949 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 421/828] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:28.550949 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82586198-b0ef-4a70-afa8-da6c3d4e0f8c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:28.568721 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd020fca-0e11-4253-9f59-b59c89e573a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:28.569489 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd020fca-0e11-4253-9f59-b59c89e573a1 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:28.569870 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd020fca-0e11-4253-9f59-b59c89e573a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:28.570181 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd020fca-0e11-4253-9f59-b59c89e573a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:28.571192 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd020fca-0e11-4253-9f59-b59c89e573a1 None admin] RBAC: Adding query filter params (role.id=4fa8f06085c741a5b3cf64e63cc0754f, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=1899c317083d40f5b490cd9eb63fe521) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:28.571550 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd020fca-0e11-4253-9f59-b59c89e573a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:28.573298 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd020fca-0e11-4253-9f59-b59c89e573a1 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:28.580697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 408/829] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:51:28 2022] GET /identity/v3/role_assignments?user.id=1899c317083d40f5b490cd9eb63fe521&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=4fa8f06085c741a5b3cf64e63cc0754f => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:28.582428 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419f623-6bb6-4ee2-af21-7dee04141ad1 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:28.583138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419f623-6bb6-4ee2-af21-7dee04141ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:28.583138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419f623-6bb6-4ee2-af21-7dee04141ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:28.595411 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:28.615288 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-c419f623-6bb6-4ee2-af21-7dee04141ad1 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:28.616560 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-c419f623-6bb6-4ee2-af21-7dee04141ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:28.680752 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82586198-b0ef-4a70-afa8-da6c3d4e0f8c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:28.697451 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 409/830] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:28.698144 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419f623-6bb6-4ee2-af21-7dee04141ad1 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:28.732900 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1f4e0-5154-4dd7-911a-8d778bfd1211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:28.733717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1f4e0-5154-4dd7-911a-8d778bfd1211 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:28.734001 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1f4e0-5154-4dd7-911a-8d778bfd1211 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:28.734321 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1f4e0-5154-4dd7-911a-8d778bfd1211 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:28.746592 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1f4e0-5154-4dd7-911a-8d778bfd1211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:28.748069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1f4e0-5154-4dd7-911a-8d778bfd1211 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:28.766521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-d4f1f4e0-5154-4dd7-911a-8d778bfd1211 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:28.766521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:28.766521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:28.766521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:28.766521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:28.766521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:28.766521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:28.766521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:28.766521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:28.766521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:28.766521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:28.766521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:28.766521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:28.766521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:28.766521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:28.768088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:28.768088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:28.768088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:28.768088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:28.768088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:28.768088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:28.768088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:28.768088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:28.768088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:28.768088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:28.768088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:28.768088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:28.768088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:28.768088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:28.768088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:28.768088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:28.768088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:28.769146 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:28.769146 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:28.769146 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:28.769146 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:28.773142 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 422/831] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:28.773951 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1f4e0-5154-4dd7-911a-8d778bfd1211 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:28.804388 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:28.825980 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419f623-6bb6-4ee2-af21-7dee04141ad1 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:28.878997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:28.881072 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38756174-ff2b-45c7-a4d7-5a39d8515b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:28.882251 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38756174-ff2b-45c7-a4d7-5a39d8515b3a None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:28.882755 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38756174-ff2b-45c7-a4d7-5a39d8515b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:28.883236 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38756174-ff2b-45c7-a4d7-5a39d8515b3a None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:28.884625 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38756174-ff2b-45c7-a4d7-5a39d8515b3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:28.885237 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38756174-ff2b-45c7-a4d7-5a39d8515b3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:28.887923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38756174-ff2b-45c7-a4d7-5a39d8515b3a None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:28.897261 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1f4e0-5154-4dd7-911a-8d778bfd1211 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:28.915325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 410/832] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:28.915325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38756174-ff2b-45c7-a4d7-5a39d8515b3a None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:28.941524 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd835b-18ff-47a9-89f9-fcecdfec2d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:28.942405 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd835b-18ff-47a9-89f9-fcecdfec2d5a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:28.942691 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd835b-18ff-47a9-89f9-fcecdfec2d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:28.942972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd835b-18ff-47a9-89f9-fcecdfec2d5a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:28.943632 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd835b-18ff-47a9-89f9-fcecdfec2d5a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:28.944999 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd835b-18ff-47a9-89f9-fcecdfec2d5a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:28.958224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c2bd835b-18ff-47a9-89f9-fcecdfec2d5a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 17:51:28.958224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:28.958224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:28.958224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:28.958224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:28.958224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:28.958224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:28.958224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:28.958224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 17:51:28.958224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:28.958224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:28.958224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:28.958224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:28.958224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:28.958224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:28.958224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:28.958224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:28.959291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:28.959291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:28.959291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:28.959291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:28.959291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:28.959291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:28.959291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:28.959291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:28.959291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:28.959291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:28.959291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:28.959291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:28.959291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:28.959291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:28.959291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:28.959291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:28.960127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:28.960127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:28.960127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:28.960127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:28.960127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:28.960127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:28.960127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:28.960127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:28.960127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:28.960127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:28.960127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:28.960127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:28.960127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 17:51:28.960127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:28.960127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 423/833] 10.222.0.76 () {56 vars in 1114 bytes} [Thu May 26 17:51:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:28.960914 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd835b-18ff-47a9-89f9-fcecdfec2d5a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:29.051037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38756174-ff2b-45c7-a4d7-5a39d8515b3a None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:29.082220 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd835b-18ff-47a9-89f9-fcecdfec2d5a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:29.090629 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c03e9-5819-4664-9d4f-7a2f4c8e9687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:29.091923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c03e9-5819-4664-9d4f-7a2f4c8e9687 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:29.092396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c03e9-5819-4664-9d4f-7a2f4c8e9687 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:29.093154 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c03e9-5819-4664-9d4f-7a2f4c8e9687 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:29.094417 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6c03e9-5819-4664-9d4f-7a2f4c8e9687 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:29.094910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6c03e9-5819-4664-9d4f-7a2f4c8e9687 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:29.097192 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6c03e9-5819-4664-9d4f-7a2f4c8e9687 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:29.113557 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 411/834] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:29.114694 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c03e9-5819-4664-9d4f-7a2f4c8e9687 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:29.120273 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:29.121193 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c7c8a-62ef-435f-8f8c-3cdf8ca946bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:29.122138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c7c8a-62ef-435f-8f8c-3cdf8ca946bf None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:29.122274 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c7c8a-62ef-435f-8f8c-3cdf8ca946bf None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:29.122703 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c7c8a-62ef-435f-8f8c-3cdf8ca946bf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:29.132384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6c7c8a-62ef-435f-8f8c-3cdf8ca946bf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:29.133883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6c7c8a-62ef-435f-8f8c-3cdf8ca946bf None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:29.149635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-3a6c7c8a-62ef-435f-8f8c-3cdf8ca946bf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 17:51:29.149635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:29.149635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:29.149635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:29.149635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:29.149635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:29.149635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:29.149635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:29.149635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:29.149635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:29.149635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:29.149635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:29.149635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:51:29.149635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:51:29.149635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:51:29.149635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:51:29.150607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:29.150607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:29.150607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:51:29.150607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:29.150607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:51:29.150607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:29.150607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:29.150607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:29.150607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:29.150607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:29.150607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:29.150607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:29.150607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:29.150607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:29.150607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:29.150607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:29.150607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:29.150607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:29.151600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:29.151600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:29.151600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:51:29.151600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:51:29.151600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:51:29.151600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:51:29.151600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:51:29.151600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:51:29.151600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 17:51:29.151600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:29.151600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 424/835] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:51:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:29.153248 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c7c8a-62ef-435f-8f8c-3cdf8ca946bf None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:29.164621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:29.245076 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c03e9-5819-4664-9d4f-7a2f4c8e9687 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:29.262351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c7c8a-62ef-435f-8f8c-3cdf8ca946bf None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:29.284361 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b00aa-2caf-411a-b586-80f39d5794f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:29.285079 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b00aa-2caf-411a-b586-80f39d5794f2 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:29.285365 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b00aa-2caf-411a-b586-80f39d5794f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:29.285642 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b00aa-2caf-411a-b586-80f39d5794f2 None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:29.286652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b00aa-2caf-411a-b586-80f39d5794f2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:29.286962 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b00aa-2caf-411a-b586-80f39d5794f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:29.288336 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b00aa-2caf-411a-b586-80f39d5794f2 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:29.293216 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 412/836] 10.222.0.76 () {56 vars in 1114 bytes} [Thu May 26 17:51:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:29.294013 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b00aa-2caf-411a-b586-80f39d5794f2 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:29.306834 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794795f-585a-419a-b37b-0c7f6e676f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:29.307963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794795f-585a-419a-b37b-0c7f6e676f39 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:29.308467 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794795f-585a-419a-b37b-0c7f6e676f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:29.308897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794795f-585a-419a-b37b-0c7f6e676f39 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:29.317319 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b794795f-585a-419a-b37b-0c7f6e676f39 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:29.318991 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b794795f-585a-419a-b37b-0c7f6e676f39 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:29.335244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-b794795f-585a-419a-b37b-0c7f6e676f39 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:51:29.335244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:29.335244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:29.335244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:29.335244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:29.335244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:29.335244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:29.335244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:29.335244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:29.335244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:29.335244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:29.335244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:29.335244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:51:29.335244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:51:29.335244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:51:29.335244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:51:29.336434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:29.336434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:29.336434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:51:29.336434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:29.336434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:51:29.336434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:29.336434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:29.336434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:29.336434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:29.336434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:29.336434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:29.336434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:29.336434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:29.336434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:29.336434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:29.336434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:29.336434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:29.336434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:29.337586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:29.337586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:29.337586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:51:29.337586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:51:29.337586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:51:29.337586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:51:29.337586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:51:29.337586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:51:29.337586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:51:29.337586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:29.339471 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 425/837] 10.222.0.76 () {56 vars in 1117 bytes} [Thu May 26 17:51:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:29.340421 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794795f-585a-419a-b37b-0c7f6e676f39 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:29.372360 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:29.410408 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b00aa-2caf-411a-b586-80f39d5794f2 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:29.420631 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:29.449306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc9f83-b56d-42be-811a-4d21016fa14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:29.449985 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc9f83-b56d-42be-811a-4d21016fa14d None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:29.450304 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc9f83-b56d-42be-811a-4d21016fa14d None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:29.450584 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc9f83-b56d-42be-811a-4d21016fa14d None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:29.451429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc9f83-b56d-42be-811a-4d21016fa14d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:29.451744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc9f83-b56d-42be-811a-4d21016fa14d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:29.453176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc9f83-b56d-42be-811a-4d21016fa14d None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:29.459911 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794795f-585a-419a-b37b-0c7f6e676f39 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:29.481560 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 413/838] 10.222.0.76 () {56 vars in 1135 bytes} [Thu May 26 17:51:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:29.482970 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc9f83-b56d-42be-811a-4d21016fa14d None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:29.503084 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b08db-c5a9-4a3f-912e-ec57685e8883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:29.504547 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b08db-c5a9-4a3f-912e-ec57685e8883 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:29.505042 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b08db-c5a9-4a3f-912e-ec57685e8883 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:29.505521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b08db-c5a9-4a3f-912e-ec57685e8883 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:29.506452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b08db-c5a9-4a3f-912e-ec57685e8883 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:29.507368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b08db-c5a9-4a3f-912e-ec57685e8883 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:29.508924 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b08db-c5a9-4a3f-912e-ec57685e8883 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:29.515359 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 426/839] 10.222.0.76 () {56 vars in 1114 bytes} [Thu May 26 17:51:29 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:29.516085 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b08db-c5a9-4a3f-912e-ec57685e8883 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:29.635047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc9f83-b56d-42be-811a-4d21016fa14d None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:29.668250 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:29.678903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b08db-c5a9-4a3f-912e-ec57685e8883 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:29.681804 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b6bd3-f8d4-499a-af8b-bcde2cb35ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:29.682811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b6bd3-f8d4-499a-af8b-bcde2cb35ffc None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:29.683277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b6bd3-f8d4-499a-af8b-bcde2cb35ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:29.683763 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b6bd3-f8d4-499a-af8b-bcde2cb35ffc None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:29.684811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b6bd3-f8d4-499a-af8b-bcde2cb35ffc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:29.685367 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b6bd3-f8d4-499a-af8b-bcde2cb35ffc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:29.687245 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b6bd3-f8d4-499a-af8b-bcde2cb35ffc None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:29.709390 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 414/840] 10.222.0.76 () {56 vars in 1149 bytes} [Thu May 26 17:51:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:29.718000 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b6bd3-f8d4-499a-af8b-bcde2cb35ffc None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:29.724661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:29.730238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d194f-f0bf-4ce9-904e-3eec6813cccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:29.731053 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d194f-f0bf-4ce9-904e-3eec6813cccb None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:29.731768 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d194f-f0bf-4ce9-904e-3eec6813cccb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:29.732050 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d194f-f0bf-4ce9-904e-3eec6813cccb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:29.747672 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d194f-f0bf-4ce9-904e-3eec6813cccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:29.749340 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d194f-f0bf-4ce9-904e-3eec6813cccb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:29.776640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-e49d194f-f0bf-4ce9-904e-3eec6813cccb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:29.776640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:29.776640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:29.776640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:29.776640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:29.776640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:29.776640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:29.776640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:29.776640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:29.776640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:29.776640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:29.776640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:29.776640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:29.776640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:29.776640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:29.777722 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:29.777722 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:29.777722 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:29.777722 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:29.777722 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:29.777722 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:29.777722 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:29.777722 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:29.777722 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:29.777722 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:29.777722 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:29.777722 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:29.777722 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:29.777722 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:29.777722 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:29.777722 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:29.777722 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:29.778680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:29.778680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:29.778680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:29.778680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:29.778680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 427/841] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:29.784739 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d194f-f0bf-4ce9-904e-3eec6813cccb None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:29.866109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b6bd3-f8d4-499a-af8b-bcde2cb35ffc None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:29.911726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d194f-f0bf-4ce9-904e-3eec6813cccb None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:29.921264 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc9153-2c71-431b-9143-66ee2b6b8ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:29.922713 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc9153-2c71-431b-9143-66ee2b6b8ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:29.923613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc9153-2c71-431b-9143-66ee2b6b8ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:29.924380 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc9153-2c71-431b-9143-66ee2b6b8ba1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:29.949858 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c8216-8fb9-40df-8c1a-139918ddafe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:29.950590 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c8216-8fb9-40df-8c1a-139918ddafe1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:29.950878 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c8216-8fb9-40df-8c1a-139918ddafe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:29.951154 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c8216-8fb9-40df-8c1a-139918ddafe1 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:29.951930 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07c8216-8fb9-40df-8c1a-139918ddafe1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:29.952182 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07c8216-8fb9-40df-8c1a-139918ddafe1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:29.953573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07c8216-8fb9-40df-8c1a-139918ddafe1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:29.974238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 428/842] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:29.975590 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c8216-8fb9-40df-8c1a-139918ddafe1 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:29.996163 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:30.017236 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc9153-2c71-431b-9143-66ee2b6b8ba1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:30.019118 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc9153-2c71-431b-9143-66ee2b6b8ba1 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:30.030305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:30.093778 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c8216-8fb9-40df-8c1a-139918ddafe1 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:30.131121 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 415/843] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:51:29 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:51:30.134430 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc9153-2c71-431b-9143-66ee2b6b8ba1 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:30.148183 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fee171-c5b8-43a2-acfd-f858af650e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:30.148992 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fee171-c5b8-43a2-acfd-f858af650e32 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:30.149276 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fee171-c5b8-43a2-acfd-f858af650e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:30.149552 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fee171-c5b8-43a2-acfd-f858af650e32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:30.159529 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fee171-c5b8-43a2-acfd-f858af650e32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:30.161755 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fee171-c5b8-43a2-acfd-f858af650e32 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:30.178057 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-45fee171-c5b8-43a2-acfd-f858af650e32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:30.178057 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:30.178057 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:30.178057 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:30.178057 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:30.178057 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:30.178057 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:30.178057 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:30.178057 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:30.178057 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:30.178057 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:30.178057 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:30.178057 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:51:30.178057 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:51:30.178057 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:51:30.179104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:51:30.179104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:30.179104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:30.179104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:30.179104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:30.179104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:30.179104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:30.179104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:30.179104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:30.179104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:30.179104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:30.179104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:30.179104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:30.179104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:30.179104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:30.179104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:30.179104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:30.180006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:51:30.180006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:51:30.180006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:51:30.180006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:51:30.180006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:51:30.180006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:51:30.180006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:30.180006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:30.186542 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 429/844] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:30.188384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fee171-c5b8-43a2-acfd-f858af650e32 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:30.232067 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:30.252652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc9153-2c71-431b-9143-66ee2b6b8ba1 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:30.288731 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee98a58-2e61-453b-8e84-9cd57a223c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:30.289535 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee98a58-2e61-453b-8e84-9cd57a223c44 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:30.289820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee98a58-2e61-453b-8e84-9cd57a223c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:30.290097 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee98a58-2e61-453b-8e84-9cd57a223c44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:30.302842 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee98a58-2e61-453b-8e84-9cd57a223c44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:30.304376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee98a58-2e61-453b-8e84-9cd57a223c44 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:30.305921 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fee171-c5b8-43a2-acfd-f858af650e32 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:30.308966 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:30.323653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-7ee98a58-2e61-453b-8e84-9cd57a223c44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:30.323653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:30.323653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:30.323653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:30.323653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:30.323653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:30.323653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:30.323653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:30.323653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:30.323653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:30.323653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:30.323653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:30.323653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:30.323653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:30.323653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:30.324617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:30.324617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:30.324617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:30.324617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:30.324617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:30.324617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:30.324617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:30.324617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:30.324617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:30.324617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:30.324617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:30.324617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:30.324617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:30.324617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:30.324617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:30.324617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:30.324617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:30.325513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:30.325513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:30.325513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:30.325513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:30.329737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 416/845] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:30.331084 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee98a58-2e61-453b-8e84-9cd57a223c44 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:30.349985 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fff504-8004-4014-8805-16eebf94d1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:30.351204 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fff504-8004-4014-8805-16eebf94d1ae None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:30.351661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fff504-8004-4014-8805-16eebf94d1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:30.352143 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fff504-8004-4014-8805-16eebf94d1ae None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:30.353111 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fff504-8004-4014-8805-16eebf94d1ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:30.353602 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fff504-8004-4014-8805-16eebf94d1ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:30.355351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fff504-8004-4014-8805-16eebf94d1ae None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:30.374532 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 430/846] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:51:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:30.375924 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fff504-8004-4014-8805-16eebf94d1ae None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:30.443112 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee98a58-2e61-453b-8e84-9cd57a223c44 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:30.482363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27133d82-245f-4250-9a5d-138ea20ff95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:30.483605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27133d82-245f-4250-9a5d-138ea20ff95d None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:30.484073 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27133d82-245f-4250-9a5d-138ea20ff95d None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:30.484993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27133d82-245f-4250-9a5d-138ea20ff95d None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:30.486255 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27133d82-245f-4250-9a5d-138ea20ff95d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:30.487358 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27133d82-245f-4250-9a5d-138ea20ff95d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:30.489848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27133d82-245f-4250-9a5d-138ea20ff95d None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:30.492474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:30.495963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fff504-8004-4014-8805-16eebf94d1ae None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:30.509974 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 417/847] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:30.517265 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27133d82-245f-4250-9a5d-138ea20ff95d None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:30.543819 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeff97c-2867-4b0e-8dd3-63bd7a85fdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:30.545477 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeff97c-2867-4b0e-8dd3-63bd7a85fdb3 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:30.545947 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeff97c-2867-4b0e-8dd3-63bd7a85fdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:30.546456 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeff97c-2867-4b0e-8dd3-63bd7a85fdb3 None admin] PATH_INFO: `/v3/projects/ca1d09f491d944f99425a641561b03c3/users/9ca774e864ca44cfb187e38bc8a8f6ea/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:30.548839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeff97c-2867-4b0e-8dd3-63bd7a85fdb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1d09f491d944f99425a641561b03c3, user_id=9ca774e864ca44cfb187e38bc8a8f6ea, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:30.551010 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeff97c-2867-4b0e-8dd3-63bd7a85fdb3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:30.574339 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 431/848] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:51:30 2022] PUT /identity/v3/projects/ca1d09f491d944f99425a641561b03c3/users/9ca774e864ca44cfb187e38bc8a8f6ea/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:51:30.665363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27133d82-245f-4250-9a5d-138ea20ff95d None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:30.711736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b7cf0-b66d-4b48-af07-11109816f4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:30.712471 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b7cf0-b66d-4b48-af07-11109816f4bd None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:30.712760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b7cf0-b66d-4b48-af07-11109816f4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:30.713047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b7cf0-b66d-4b48-af07-11109816f4bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:30.721537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b7cf0-b66d-4b48-af07-11109816f4bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:30.723106 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b7cf0-b66d-4b48-af07-11109816f4bd None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:30.740468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-c62b7cf0-b66d-4b48-af07-11109816f4bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:30.740468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:30.740468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:30.740468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:30.740468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:30.740468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:30.740468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:30.740468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:30.740468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:30.740468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:30.740468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:30.740468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:30.740468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:51:30.740468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:51:30.740468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:51:30.741479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:51:30.741479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:30.741479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:30.741479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:30.741479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:30.741479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:30.741479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:30.741479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:30.741479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:30.741479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:30.741479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:30.741479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:30.741479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:30.741479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:30.741479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:30.741479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:30.741479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:30.742463 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:51:30.742463 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:51:30.742463 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:51:30.742463 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:51:30.742463 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:51:30.742463 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:51:30.742463 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:30.742463 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:30.746352 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 418/849] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:30.751416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeff97c-2867-4b0e-8dd3-63bd7a85fdb3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:30.787062 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:30.886569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeff97c-2867-4b0e-8dd3-63bd7a85fdb3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:30.925712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6389f7-4cae-481d-b19d-8ea4484e2c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:30.926447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6389f7-4cae-481d-b19d-8ea4484e2c80 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:30.926737 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6389f7-4cae-481d-b19d-8ea4484e2c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:30.927018 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6389f7-4cae-481d-b19d-8ea4484e2c80 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:30.927839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6389f7-4cae-481d-b19d-8ea4484e2c80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:30.928156 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6389f7-4cae-481d-b19d-8ea4484e2c80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:30.929681 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6389f7-4cae-481d-b19d-8ea4484e2c80 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:30.950079 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 432/850] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:51:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:30.951656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b7cf0-b66d-4b48-af07-11109816f4bd None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:31.007875 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e1391-abf8-4db3-a483-9295bb03d048 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:31.008163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e1391-abf8-4db3-a483-9295bb03d048 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:31.008448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e1391-abf8-4db3-a483-9295bb03d048 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:31.009839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 433/851] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:31.015773 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fb221-b5cd-431e-9539-1a1814afa7db None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:31.016103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fb221-b5cd-431e-9539-1a1814afa7db None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:31.016328 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fb221-b5cd-431e-9539-1a1814afa7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:31.022310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:31.035443 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-393fb221-b5cd-431e-9539-1a1814afa7db None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:31.037157 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-393fb221-b5cd-431e-9539-1a1814afa7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:31.072775 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b7cf0-b66d-4b48-af07-11109816f4bd None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:31.114179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cc9ef-bd51-43e9-8363-82c2f28cb25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:31.114964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cc9ef-bd51-43e9-8363-82c2f28cb25e None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:31.115253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cc9ef-bd51-43e9-8363-82c2f28cb25e None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:31.115536 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cc9ef-bd51-43e9-8363-82c2f28cb25e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:31.116508 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17cc9ef-bd51-43e9-8363-82c2f28cb25e None admin] RBAC: Adding query filter params (role.id=e776303775a2423aa0c62c161ecb5a7e, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=3e06d1b0e1a9472db764f4be7e42fdd9) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:31.116936 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17cc9ef-bd51-43e9-8363-82c2f28cb25e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:31.118573 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17cc9ef-bd51-43e9-8363-82c2f28cb25e None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:31.131521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 434/853] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) May 26 17:51:31.131713 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 419/853] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:51:30 2022] GET /identity/v3/role_assignments?user.id=3e06d1b0e1a9472db764f4be7e42fdd9&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=e776303775a2423aa0c62c161ecb5a7e => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:31.153873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fb221-b5cd-431e-9539-1a1814afa7db None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:31.169372 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:31.274664 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fb221-b5cd-431e-9539-1a1814afa7db None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:31.315957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb492daf-c119-4fe3-9750-5fa7c435fb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:31.316800 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb492daf-c119-4fe3-9750-5fa7c435fb30 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:31.317143 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb492daf-c119-4fe3-9750-5fa7c435fb30 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:31.317485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb492daf-c119-4fe3-9750-5fa7c435fb30 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:31.318179 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb492daf-c119-4fe3-9750-5fa7c435fb30 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:31.319740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb492daf-c119-4fe3-9750-5fa7c435fb30 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:31.333658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-fb492daf-c119-4fe3-9750-5fa7c435fb30 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 17:51:31.333658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:31.333658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:31.333658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:31.333658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:31.333658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:31.333658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:31.333658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:31.333658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 17:51:31.333658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:31.333658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:31.333658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:31.333658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:31.333658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:31.333658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:31.333658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:31.334743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:31.334743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:31.334743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:31.334743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:31.334743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:31.334743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:31.334743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:31.334743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:31.334743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:31.334743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:31.334743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:31.334743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:31.334743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:31.334743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:31.334743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:31.334743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:31.335574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:31.335574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:31.335574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:31.335574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:31.335574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:31.335574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:31.335574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:31.335574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:31.335574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:31.335574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:31.335574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:31.335574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:31.335574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:31.335574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 17:51:31.335574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:31.335574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 435/854] 10.222.0.76 () {56 vars in 1132 bytes} [Thu May 26 17:51:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:31.343371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cc9ef-bd51-43e9-8363-82c2f28cb25e None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:31.462245 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cc9ef-bd51-43e9-8363-82c2f28cb25e None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:31.482664 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:31.504142 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa101a-2374-4467-b636-54b0724579ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:31.504995 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa101a-2374-4467-b636-54b0724579ef None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:31.505345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa101a-2374-4467-b636-54b0724579ef None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:31.505646 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa101a-2374-4467-b636-54b0724579ef None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:31.506478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa101a-2374-4467-b636-54b0724579ef None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:31.506822 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa101a-2374-4467-b636-54b0724579ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:31.508415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa101a-2374-4467-b636-54b0724579ef None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:31.515060 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 420/855] 10.222.0.76 () {56 vars in 1126 bytes} [Thu May 26 17:51:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:31.524533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb492daf-c119-4fe3-9750-5fa7c435fb30 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:31.614447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:31.637591 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb492daf-c119-4fe3-9750-5fa7c435fb30 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:31.679878 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9385b-0837-45f7-9e23-20b3df9951c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:31.680555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9385b-0837-45f7-9e23-20b3df9951c7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:31.680838 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9385b-0837-45f7-9e23-20b3df9951c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:31.681114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9385b-0837-45f7-9e23-20b3df9951c7 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:31.681821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9385b-0837-45f7-9e23-20b3df9951c7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:31.682126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9385b-0837-45f7-9e23-20b3df9951c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:31.683525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9385b-0837-45f7-9e23-20b3df9951c7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:31.689030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 436/856] 10.222.0.76 () {56 vars in 1126 bytes} [Thu May 26 17:51:31 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:31.846524 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5240f06-00be-4f6e-85cd-c052b77b84bb None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:31.847036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5240f06-00be-4f6e-85cd-c052b77b84bb None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:31.847451 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5240f06-00be-4f6e-85cd-c052b77b84bb None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:31.849133 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 421/857] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:31.856024 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bab5c-de30-48e1-a6e6-90d96ea8bf2b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:31.856463 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bab5c-de30-48e1-a6e6-90d96ea8bf2b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:31.856846 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bab5c-de30-48e1-a6e6-90d96ea8bf2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:31.876230 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e04bab5c-de30-48e1-a6e6-90d96ea8bf2b None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:31.877645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e04bab5c-de30-48e1-a6e6-90d96ea8bf2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:31.954112 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 437/858] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:31.979052 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5240f06-00be-4f6e-85cd-c052b77b84bb None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:32.093623 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5240f06-00be-4f6e-85cd-c052b77b84bb None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:32.135559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f35674-fc07-4c5f-a3c4-f66553882794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:32.136197 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f35674-fc07-4c5f-a3c4-f66553882794 None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:32.136478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f35674-fc07-4c5f-a3c4-f66553882794 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:32.136755 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f35674-fc07-4c5f-a3c4-f66553882794 None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:32.137498 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f35674-fc07-4c5f-a3c4-f66553882794 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:32.138912 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f35674-fc07-4c5f-a3c4-f66553882794 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:32.159141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 422/859] 10.222.0.76 () {60 vars in 1141 bytes} [Thu May 26 17:51:31 2022] POST /identity/v3/services => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:51:33.026806 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fa413-5f90-459b-95ae-93021f232663 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:33.027830 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fa413-5f90-459b-95ae-93021f232663 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:33.028348 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fa413-5f90-459b-95ae-93021f232663 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:33.030236 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 438/860] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:33.036429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a7748-6af4-4c5a-af0a-179ebcff32ef None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:33.037328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a7748-6af4-4c5a-af0a-179ebcff32ef None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:33.037809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a7748-6af4-4c5a-af0a-179ebcff32ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:33.056083 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-c59a7748-6af4-4c5a-af0a-179ebcff32ef None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:33.057635 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-c59a7748-6af4-4c5a-af0a-179ebcff32ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:33.096494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:33.145232 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 423/861] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:33.168448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62fa413-5f90-459b-95ae-93021f232663 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:33.245141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:33.277986 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62fa413-5f90-459b-95ae-93021f232663 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:33.313835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dbbfe-8e98-4be9-948a-dda08e2af8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:33.315404 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dbbfe-8e98-4be9-948a-dda08e2af8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:33.315694 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dbbfe-8e98-4be9-948a-dda08e2af8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:33.315965 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dbbfe-8e98-4be9-948a-dda08e2af8b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:33.326660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23dbbfe-8e98-4be9-948a-dda08e2af8b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:51:33.334605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-f23dbbfe-8e98-4be9-948a-dda08e2af8b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:51:33.334605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:33.334605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:33.334605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:33.334605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:33.334605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:33.334605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:33.334605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:33.334605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:33.334605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:33.334605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:33.334605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:33.334605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:51:33.334605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:51:33.334605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:51:33.334605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:51:33.335626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:51:33.335626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:51:33.335626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:51:33.335626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:51:33.335626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:33.335626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:33.335626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:33.335626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:33.335626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:33.335626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:33.335626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:33.335626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:33.335626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:33.335626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:33.335626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:33.335626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:33.336455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:33.336455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:33.336455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:33.336455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:33.336455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:51:33.336455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:51:33.336455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:51:33.336455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:51:33.336455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:51:33.336455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:51:33.336455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:51:33.336455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:33.336455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 439/862] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:51:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:33.342033 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a7748-6af4-4c5a-af0a-179ebcff32ef None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:33.459579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a7748-6af4-4c5a-af0a-179ebcff32ef None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:33.497826 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76f137-bf23-492a-b50c-3c1b086ccf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:33.498889 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76f137-bf23-492a-b50c-3c1b086ccf88 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:33.499397 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76f137-bf23-492a-b50c-3c1b086ccf88 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:33.499904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76f137-bf23-492a-b50c-3c1b086ccf88 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:33.500917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76f137-bf23-492a-b50c-3c1b086ccf88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:33.501668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76f137-bf23-492a-b50c-3c1b086ccf88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:33.505166 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76f137-bf23-492a-b50c-3c1b086ccf88 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:33.518768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 424/863] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:51:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:33.524662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dbbfe-8e98-4be9-948a-dda08e2af8b4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:33.608752 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27f2b7-0618-4de8-977a-10246aea8843 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:33.609140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27f2b7-0618-4de8-977a-10246aea8843 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:33.609344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27f2b7-0618-4de8-977a-10246aea8843 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:33.613400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 425/864] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:51:33.623999 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a0220-8bc1-454d-b2c7-13fd2c469514 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:33.624473 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a0220-8bc1-454d-b2c7-13fd2c469514 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:33.624853 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a0220-8bc1-454d-b2c7-13fd2c469514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:33.643654 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-ab9a0220-8bc1-454d-b2c7-13fd2c469514 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:33.645171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-ab9a0220-8bc1-454d-b2c7-13fd2c469514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:33.667920 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dbbfe-8e98-4be9-948a-dda08e2af8b4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:33.673239 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:33.703495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:33.710184 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79641235-fa80-496e-8fb8-547cd4750e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:33.711006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79641235-fa80-496e-8fb8-547cd4750e91 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:33.711290 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79641235-fa80-496e-8fb8-547cd4750e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:33.711567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79641235-fa80-496e-8fb8-547cd4750e91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:33.725077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79641235-fa80-496e-8fb8-547cd4750e91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:33.726690 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79641235-fa80-496e-8fb8-547cd4750e91 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:33.740074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 426/865] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:33.753285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-79641235-fa80-496e-8fb8-547cd4750e91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:33.753285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:33.753285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:33.753285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:33.753285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:33.753285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:33.753285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:33.753285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:33.753285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:33.753285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:33.753285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:33.753285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:33.753285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:33.753285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:33.753285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:33.754201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:33.754201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:33.754201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:33.754201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:33.754201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:33.754201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:33.754201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:33.754201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:33.754201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:33.754201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:33.754201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:33.754201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:33.754201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:33.754201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:33.754201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:33.754201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:33.754201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:33.755104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:33.755104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:33.755104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:33.755104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:33.757305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 440/866] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:33.762658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a0220-8bc1-454d-b2c7-13fd2c469514 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:33.763987 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79641235-fa80-496e-8fb8-547cd4750e91 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:33.879200 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a0220-8bc1-454d-b2c7-13fd2c469514 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:33.881451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79641235-fa80-496e-8fb8-547cd4750e91 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:33.917248 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42dc62a-ea76-483f-a650-5936db22c516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:33.918050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dc62a-ea76-483f-a650-5936db22c516 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:33.918435 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dc62a-ea76-483f-a650-5936db22c516 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:33.918731 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dc62a-ea76-483f-a650-5936db22c516 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:33.919556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42dc62a-ea76-483f-a650-5936db22c516 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:33.919868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42dc62a-ea76-483f-a650-5936db22c516 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:33.920364 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e2111-0385-4853-a406-4260f7eafc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:33.921249 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42dc62a-ea76-483f-a650-5936db22c516 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:33.921580 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e2111-0385-4853-a406-4260f7eafc9d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:33.922108 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e2111-0385-4853-a406-4260f7eafc9d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:33.922594 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e2111-0385-4853-a406-4260f7eafc9d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:33.934258 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e2111-0385-4853-a406-4260f7eafc9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:51:33.938469 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 427/867] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:33.940031 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42dc62a-ea76-483f-a650-5936db22c516 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:33.943716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-639e2111-0385-4853-a406-4260f7eafc9d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:51:33.943716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:33.943716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:33.943716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:33.943716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:33.943716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:33.943716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:33.943716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:33.943716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:33.943716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:33.943716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:33.943716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:33.943716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:51:33.943716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:51:33.943716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:51:33.943716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:51:33.944679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:51:33.944679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:51:33.944679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:51:33.944679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:51:33.944679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:33.944679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:33.944679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:33.944679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:33.944679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:33.944679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:33.944679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:33.944679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:33.944679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:33.944679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:33.944679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:33.944679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:33.945522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:33.945522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:33.945522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:33.945522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:33.945522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:51:33.945522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:51:33.945522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:51:33.945522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:51:33.945522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:51:33.945522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:51:33.945522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:51:33.945522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:33.952926 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 441/868] 10.222.0.76 () {56 vars in 1120 bytes} [Thu May 26 17:51:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:33.954066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e2111-0385-4853-a406-4260f7eafc9d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:33.964801 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:34.005493 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:34.051410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42dc62a-ea76-483f-a650-5936db22c516 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:34.070495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e2111-0385-4853-a406-4260f7eafc9d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:34.087776 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc66d0d-10ac-40a0-a4d6-7796f7340fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:34.088576 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc66d0d-10ac-40a0-a4d6-7796f7340fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:34.088851 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc66d0d-10ac-40a0-a4d6-7796f7340fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:34.089129 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc66d0d-10ac-40a0-a4d6-7796f7340fc9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:34.098167 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc66d0d-10ac-40a0-a4d6-7796f7340fc9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:34.099664 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc66d0d-10ac-40a0-a4d6-7796f7340fc9 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:34.109282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c265f6c-7a6c-4626-b192-2b0ad95a1007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:34.110283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c265f6c-7a6c-4626-b192-2b0ad95a1007 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:34.110812 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c265f6c-7a6c-4626-b192-2b0ad95a1007 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:34.111320 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c265f6c-7a6c-4626-b192-2b0ad95a1007 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:34.112355 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c265f6c-7a6c-4626-b192-2b0ad95a1007 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:34.112859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c265f6c-7a6c-4626-b192-2b0ad95a1007 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:34.114374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-2fc66d0d-10ac-40a0-a4d6-7796f7340fc9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 17:51:34.114374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:34.114374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:34.114374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:34.114374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:34.114374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:34.114374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:34.114374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:34.114374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:34.114374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:34.114374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:34.114374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:34.114374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:51:34.114374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:51:34.114374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:51:34.114374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:51:34.115280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:34.115280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:34.115280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:51:34.115280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:34.115280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:51:34.115280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:34.115280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:34.115280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:34.115280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:34.115280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:34.115280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:34.115280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:34.115280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:34.115280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:34.115280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:34.115280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:34.115280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:34.115280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:34.116223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:34.116223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:34.116223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:51:34.116223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:51:34.116223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:51:34.116223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:51:34.116223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:51:34.116223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:51:34.116223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 17:51:34.116223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:34.116223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 428/869] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:51:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:34.117129 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b455a-1a3f-4229-85fd-4696dafa2757 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:34.117217 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b455a-1a3f-4229-85fd-4696dafa2757 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:34.117439 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b455a-1a3f-4229-85fd-4696dafa2757 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:34.117580 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c265f6c-7a6c-4626-b192-2b0ad95a1007 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:34.119036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 429/870] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:34.122215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b455a-1a3f-4229-85fd-4696dafa2757 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:34.136573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 442/871] 10.222.0.76 () {56 vars in 1115 bytes} [Thu May 26 17:51:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:34.138230 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d82e89-7c15-4459-ace3-19d4a242fbc6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:34.138666 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d82e89-7c15-4459-ace3-19d4a242fbc6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:34.139078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d82e89-7c15-4459-ace3-19d4a242fbc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:34.162593 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-94d82e89-7c15-4459-ace3-19d4a242fbc6 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:34.163790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-94d82e89-7c15-4459-ace3-19d4a242fbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:34.224069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:34.246219 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 443/872] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:34 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:34.246219 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d82e89-7c15-4459-ace3-19d4a242fbc6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:34.259879 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b455a-1a3f-4229-85fd-4696dafa2757 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:34.280987 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:34.304231 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a4ef9-df3e-4982-9738-88eb73034543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:34.305034 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a4ef9-df3e-4982-9738-88eb73034543 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:34.305231 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a4ef9-df3e-4982-9738-88eb73034543 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:34.305481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a4ef9-df3e-4982-9738-88eb73034543 None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:34.306380 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a4ef9-df3e-4982-9738-88eb73034543 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:34.306696 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a4ef9-df3e-4982-9738-88eb73034543 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:34.308224 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a4ef9-df3e-4982-9738-88eb73034543 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:34.328612 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 430/873] 10.222.0.76 () {56 vars in 1135 bytes} [Thu May 26 17:51:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:34.329446 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a4ef9-df3e-4982-9738-88eb73034543 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:34.365851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d82e89-7c15-4459-ace3-19d4a242fbc6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:34.403617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc23d11-c2f8-4154-b9a4-acbcfcd03521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:34.404327 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc23d11-c2f8-4154-b9a4-acbcfcd03521 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:34.404608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc23d11-c2f8-4154-b9a4-acbcfcd03521 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:34.404880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc23d11-c2f8-4154-b9a4-acbcfcd03521 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:34.415987 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc23d11-c2f8-4154-b9a4-acbcfcd03521 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:34.417817 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc23d11-c2f8-4154-b9a4-acbcfcd03521 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:34.436305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-9bc23d11-c2f8-4154-b9a4-acbcfcd03521 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:34.436305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:34.436305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:34.436305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:34.436305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:34.436305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:34.436305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:34.436305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:34.436305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:34.436305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:34.436305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:34.436305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:34.436305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:34.436305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:34.436305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:34.437293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:34.437293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:34.437293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:34.437293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:34.437293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:34.437293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:34.437293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:34.437293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:34.437293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:34.437293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:34.437293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:34.437293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:34.437293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:34.437293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:34.437293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:34.437293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:34.437293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:34.438218 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:34.438218 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:34.438218 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:34.438218 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:34.438218 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 444/874] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:34.438598 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc23d11-c2f8-4154-b9a4-acbcfcd03521 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:34.447336 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a4ef9-df3e-4982-9738-88eb73034543 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:34.487101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3389d-c59b-48f8-94d9-48390cb56ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:34.488065 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3389d-c59b-48f8-94d9-48390cb56ba9 None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:34.488065 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3389d-c59b-48f8-94d9-48390cb56ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:34.488364 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3389d-c59b-48f8-94d9-48390cb56ba9 None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:34.489047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3389d-c59b-48f8-94d9-48390cb56ba9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:34.490508 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3389d-c59b-48f8-94d9-48390cb56ba9 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:34.500641 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:34.522609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 431/875] 10.222.0.76 () {60 vars in 1141 bytes} [Thu May 26 17:51:34 2022] POST /identity/v3/services => generated 269 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:51:34.523623 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3389d-c59b-48f8-94d9-48390cb56ba9 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:34.563570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:34.582165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc23d11-c2f8-4154-b9a4-acbcfcd03521 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:34.633963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38038dc-dbad-4410-9f78-4d56199a648d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:34.635019 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38038dc-dbad-4410-9f78-4d56199a648d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:34.635507 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38038dc-dbad-4410-9f78-4d56199a648d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:34.636021 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38038dc-dbad-4410-9f78-4d56199a648d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:34.654891 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3389d-c59b-48f8-94d9-48390cb56ba9 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:34.695818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889eb435-5e7e-4d0e-8fa1-a48b2ea5be85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:34.696511 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889eb435-5e7e-4d0e-8fa1-a48b2ea5be85 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:34.696798 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889eb435-5e7e-4d0e-8fa1-a48b2ea5be85 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:34.697078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889eb435-5e7e-4d0e-8fa1-a48b2ea5be85 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:34.697818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889eb435-5e7e-4d0e-8fa1-a48b2ea5be85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:34.698130 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889eb435-5e7e-4d0e-8fa1-a48b2ea5be85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:34.699620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889eb435-5e7e-4d0e-8fa1-a48b2ea5be85 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:34.712597 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 432/876] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:34.714222 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d365ce-fcae-45b7-9602-5fd1a0fc9ada None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:34.714222 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d365ce-fcae-45b7-9602-5fd1a0fc9ada None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:34.714543 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d365ce-fcae-45b7-9602-5fd1a0fc9ada None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:34.715797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 433/877] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:34.718395 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d365ce-fcae-45b7-9602-5fd1a0fc9ada None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:34.726256 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38038dc-dbad-4410-9f78-4d56199a648d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:34.727710 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38038dc-dbad-4410-9f78-4d56199a648d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:34.790955 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:34.825855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:34.844270 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d365ce-fcae-45b7-9602-5fd1a0fc9ada None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:34.848572 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 445/878] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:51:34 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:51:34.849605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ead7a-c046-4afc-8220-4e8ffd5ad6c6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:34.849895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ead7a-c046-4afc-8220-4e8ffd5ad6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:34.850173 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ead7a-c046-4afc-8220-4e8ffd5ad6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:34.868495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-333ead7a-c046-4afc-8220-4e8ffd5ad6c6 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:34.869719 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-333ead7a-c046-4afc-8220-4e8ffd5ad6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:34.889541 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc854650-b3bb-4647-9812-2fe0e670caab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:34.890419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc854650-b3bb-4647-9812-2fe0e670caab None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:34.890697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc854650-b3bb-4647-9812-2fe0e670caab None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:34.890978 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc854650-b3bb-4647-9812-2fe0e670caab None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:34.900560 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc854650-b3bb-4647-9812-2fe0e670caab None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:34.902138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc854650-b3bb-4647-9812-2fe0e670caab None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:34.919972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-bc854650-b3bb-4647-9812-2fe0e670caab None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:51:34.919972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:34.919972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:34.919972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:34.919972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:34.919972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:34.919972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:34.919972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:34.919972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:34.919972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:34.919972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:34.919972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:34.919972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:51:34.919972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:51:34.919972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:51:34.919972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:51:34.920973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:34.920973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:34.920973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:51:34.920973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:34.920973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:51:34.920973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:34.920973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:34.920973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:34.920973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:34.920973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:34.920973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:34.920973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:34.920973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:34.920973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:34.920973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:34.920973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:34.920973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:34.920973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:34.921939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:34.921939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:34.921939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:51:34.921939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:51:34.921939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:51:34.921939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:51:34.921939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:51:34.921939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:51:34.921939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:51:34.921939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:34.921939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 434/879] 10.222.0.76 () {56 vars in 1117 bytes} [Thu May 26 17:51:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:34.921939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc854650-b3bb-4647-9812-2fe0e670caab None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:34.958743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 446/880] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:34.959455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ead7a-c046-4afc-8220-4e8ffd5ad6c6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:35.049747 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc854650-b3bb-4647-9812-2fe0e670caab None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:35.077605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:35.082798 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ead7a-c046-4afc-8220-4e8ffd5ad6c6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:35.099790 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186394bb-6a1e-4f0c-950e-fac70b4286ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:35.100837 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:35.101079 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186394bb-6a1e-4f0c-950e-fac70b4286ae None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:35.101377 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186394bb-6a1e-4f0c-950e-fac70b4286ae None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:35.101655 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186394bb-6a1e-4f0c-950e-fac70b4286ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:35.115549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186394bb-6a1e-4f0c-950e-fac70b4286ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:35.117060 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186394bb-6a1e-4f0c-950e-fac70b4286ae None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:35.133878 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f864b6-96ef-4053-b46a-84ae10cea77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:35.134607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f864b6-96ef-4053-b46a-84ae10cea77f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:35.134890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f864b6-96ef-4053-b46a-84ae10cea77f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:35.135166 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f864b6-96ef-4053-b46a-84ae10cea77f None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:35.136036 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f864b6-96ef-4053-b46a-84ae10cea77f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:35.136347 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f864b6-96ef-4053-b46a-84ae10cea77f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:35.140272 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f864b6-96ef-4053-b46a-84ae10cea77f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:35.143329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-186394bb-6a1e-4f0c-950e-fac70b4286ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:35.143329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:35.143329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:35.143329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:35.143329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:35.143329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:35.143329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:35.143329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:35.143329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:35.143329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:35.143329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:35.143329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:35.143329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:35.143329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:35.143329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:35.144770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:35.144770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:35.144770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:35.144770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:35.144770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:35.144770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:35.144770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:35.144770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:35.144770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:35.144770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:35.144770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:35.144770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:35.144770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:35.144770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:35.144770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:35.144770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:35.144770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:35.145650 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:35.145650 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:35.145650 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:35.145650 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:35.149575 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 435/881] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:35.150594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186394bb-6a1e-4f0c-950e-fac70b4286ae None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:35.161989 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 447/882] 10.222.0.76 () {56 vars in 1149 bytes} [Thu May 26 17:51:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:35.163033 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f864b6-96ef-4053-b46a-84ae10cea77f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:35.272478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186394bb-6a1e-4f0c-950e-fac70b4286ae None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:35.274075 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f864b6-96ef-4053-b46a-84ae10cea77f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:35.315139 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3febf-ea56-4c44-818c-3c221c135913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:35.315967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3febf-ea56-4c44-818c-3c221c135913 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:35.316258 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3febf-ea56-4c44-818c-3c221c135913 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:35.316539 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3febf-ea56-4c44-818c-3c221c135913 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:35.317115 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c8534-acbf-4e03-a719-bac17ebca501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:35.317228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f3febf-ea56-4c44-818c-3c221c135913 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:35.317787 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c8534-acbf-4e03-a719-bac17ebca501 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:35.318070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c8534-acbf-4e03-a719-bac17ebca501 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:35.318377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c8534-acbf-4e03-a719-bac17ebca501 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:35.318632 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f3febf-ea56-4c44-818c-3c221c135913 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:35.319099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334c8534-acbf-4e03-a719-bac17ebca501 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:35.319407 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334c8534-acbf-4e03-a719-bac17ebca501 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:35.320773 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334c8534-acbf-4e03-a719-bac17ebca501 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:35.333159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-93f3febf-ea56-4c44-818c-3c221c135913 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 17:51:35.333159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:35.333159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:35.333159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:35.333159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:35.333159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:35.333159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:35.333159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:35.333159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 17:51:35.333159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:35.333159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:35.333159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:35.333159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:35.333159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:35.333159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:35.333159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:35.333159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:35.340414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:35.340414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:35.340414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:35.340414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:35.340414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:35.340414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:35.340414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:35.340414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:35.340414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:35.340414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:35.340414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:35.340414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:35.340414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:35.340414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:35.340414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:35.340414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:35.341312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:35.341312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:35.341312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:35.341312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:35.341312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:35.341312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:35.341312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:35.341312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:35.341312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:35.341312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:35.341312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:35.341312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:35.341312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 17:51:35.341312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:35.341312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 436/883] 10.222.0.76 () {56 vars in 1114 bytes} [Thu May 26 17:51:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:35.342236 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3febf-ea56-4c44-818c-3c221c135913 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:35.342311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 448/884] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:35.342311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c8534-acbf-4e03-a719-bac17ebca501 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:35.381969 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:35.390021 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:35.467584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c8534-acbf-4e03-a719-bac17ebca501 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:35.485894 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3febf-ea56-4c44-818c-3c221c135913 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:35.521309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82197ad8-d70d-4d9c-9dda-8e9d9da76710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:35.523757 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82197ad8-d70d-4d9c-9dda-8e9d9da76710 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:35.525126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82197ad8-d70d-4d9c-9dda-8e9d9da76710 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:35.525489 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82197ad8-d70d-4d9c-9dda-8e9d9da76710 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:35.536620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82197ad8-d70d-4d9c-9dda-8e9d9da76710 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:35.538264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82197ad8-d70d-4d9c-9dda-8e9d9da76710 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:35.538835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c457dbb-f40f-4383-987d-f329976005b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:35.539504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c457dbb-f40f-4383-987d-f329976005b9 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:35.539815 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c457dbb-f40f-4383-987d-f329976005b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:35.540073 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c457dbb-f40f-4383-987d-f329976005b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:35.556521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-82197ad8-d70d-4d9c-9dda-8e9d9da76710 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:35.556521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:35.556521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:35.556521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:35.556521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:35.556521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:35.556521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:35.556521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:35.556521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:35.556521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:35.556521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:35.556521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:35.556521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:51:35.556521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:51:35.556521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:51:35.557511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:51:35.557511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:35.557511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:35.557511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:35.557511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:35.557511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:35.557511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:35.557511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:35.557511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:35.557511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:35.557511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:35.557511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:35.557511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:35.557511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:35.557511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:35.557511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:35.557511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:35.558478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:51:35.558478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:51:35.558478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:51:35.558478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:51:35.558478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:51:35.558478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:51:35.558478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:35.558478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:35.558478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 449/885] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:35.559669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82197ad8-d70d-4d9c-9dda-8e9d9da76710 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:35.634352 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c457dbb-f40f-4383-987d-f329976005b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:35.639841 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c457dbb-f40f-4383-987d-f329976005b9 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:35.680809 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:35.693314 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82197ad8-d70d-4d9c-9dda-8e9d9da76710 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:35.693965 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:35.750247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631995d7-0f76-4a3f-a418-fbde2b336196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:35.750937 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631995d7-0f76-4a3f-a418-fbde2b336196 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:35.751217 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631995d7-0f76-4a3f-a418-fbde2b336196 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:35.751501 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631995d7-0f76-4a3f-a418-fbde2b336196 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:35.753389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631995d7-0f76-4a3f-a418-fbde2b336196 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:35.753701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631995d7-0f76-4a3f-a418-fbde2b336196 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:35.755250 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631995d7-0f76-4a3f-a418-fbde2b336196 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:35.766313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 450/886] 10.222.0.76 () {56 vars in 1114 bytes} [Thu May 26 17:51:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:35.766313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631995d7-0f76-4a3f-a418-fbde2b336196 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:35.807162 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 437/887] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:51:35 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:51:35.807951 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c457dbb-f40f-4383-987d-f329976005b9 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:35.899969 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631995d7-0f76-4a3f-a418-fbde2b336196 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:35.943976 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c457dbb-f40f-4383-987d-f329976005b9 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:35.948016 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c7f38-efa9-4a15-ad47-0e32b1ac808b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:35.948966 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c7f38-efa9-4a15-ad47-0e32b1ac808b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:35.949450 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c7f38-efa9-4a15-ad47-0e32b1ac808b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:35.950142 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c7f38-efa9-4a15-ad47-0e32b1ac808b None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:35.951260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c7f38-efa9-4a15-ad47-0e32b1ac808b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:35.951720 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c7f38-efa9-4a15-ad47-0e32b1ac808b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:35.953598 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c7f38-efa9-4a15-ad47-0e32b1ac808b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:35.971497 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 451/888] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:51:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:35.972809 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c7f38-efa9-4a15-ad47-0e32b1ac808b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:35.993725 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e5c6e-0640-4026-a5e7-67ecb0092d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:35.994698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e5c6e-0640-4026-a5e7-67ecb0092d20 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:35.995152 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e5c6e-0640-4026-a5e7-67ecb0092d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:35.995392 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e5c6e-0640-4026-a5e7-67ecb0092d20 None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:35.996322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e5c6e-0640-4026-a5e7-67ecb0092d20 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:35.997314 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e5c6e-0640-4026-a5e7-67ecb0092d20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:35.997864 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e5c6e-0640-4026-a5e7-67ecb0092d20 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:35.999490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:36.008119 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:36.010801 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 438/889] 10.222.0.76 () {56 vars in 1114 bytes} [Thu May 26 17:51:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:36.011949 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e5c6e-0640-4026-a5e7-67ecb0092d20 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:36.123844 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c7f38-efa9-4a15-ad47-0e32b1ac808b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:36.166341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e5c6e-0640-4026-a5e7-67ecb0092d20 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:36.183760 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982247fb-bfb2-4bb3-9581-a2e30b8129e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:36.184567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982247fb-bfb2-4bb3-9581-a2e30b8129e8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:36.184847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982247fb-bfb2-4bb3-9581-a2e30b8129e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:36.185118 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982247fb-bfb2-4bb3-9581-a2e30b8129e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:36.198386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982247fb-bfb2-4bb3-9581-a2e30b8129e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:36.199872 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982247fb-bfb2-4bb3-9581-a2e30b8129e8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:36.203775 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92855ff6-f36e-4848-8f38-d0d708cdc3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:36.204667 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92855ff6-f36e-4848-8f38-d0d708cdc3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:36.205321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92855ff6-f36e-4848-8f38-d0d708cdc3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:36.205753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92855ff6-f36e-4848-8f38-d0d708cdc3e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:36.207244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92855ff6-f36e-4848-8f38-d0d708cdc3e1 None admin] RBAC: Adding query filter params (role.id=4fa8f06085c741a5b3cf64e63cc0754f, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=9ca774e864ca44cfb187e38bc8a8f6ea) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:36.207735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92855ff6-f36e-4848-8f38-d0d708cdc3e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:36.209795 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92855ff6-f36e-4848-8f38-d0d708cdc3e1 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:36.216764 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 439/890] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:51:36 2022] GET /identity/v3/role_assignments?user.id=9ca774e864ca44cfb187e38bc8a8f6ea&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=4fa8f06085c741a5b3cf64e63cc0754f => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:36.217602 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92855ff6-f36e-4848-8f38-d0d708cdc3e1 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:36.219132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-982247fb-bfb2-4bb3-9581-a2e30b8129e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:36.219132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:36.219132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:36.219132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:36.219132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:36.219132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:36.219132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:36.219132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:36.219132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:36.219132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:36.219132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:36.219132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:36.219132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:36.219132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:36.219132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:36.220029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:36.220029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:36.220029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:36.220029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:36.220029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:36.220029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:36.220029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:36.220029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:36.220029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:36.220029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:36.220029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:36.220029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:36.220029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:36.220029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:36.220029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:36.220029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:36.220029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:36.220982 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:36.220982 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:36.220982 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:36.220982 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:36.222647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 452/891] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:36.229085 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982247fb-bfb2-4bb3-9581-a2e30b8129e8 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:36.326683 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:36.345232 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:36.388801 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92855ff6-f36e-4848-8f38-d0d708cdc3e1 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:36.395359 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982247fb-bfb2-4bb3-9581-a2e30b8129e8 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:36.451654 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc050810-c22e-4c9b-b7aa-355413b4727e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:36.451654 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc050810-c22e-4c9b-b7aa-355413b4727e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:36.451654 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc050810-c22e-4c9b-b7aa-355413b4727e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:36.452070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc050810-c22e-4c9b-b7aa-355413b4727e None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:36.452155 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce107478-08dc-44ac-b0a5-f713988f7542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:36.453231 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce107478-08dc-44ac-b0a5-f713988f7542 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:36.453335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc050810-c22e-4c9b-b7aa-355413b4727e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:36.453409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce107478-08dc-44ac-b0a5-f713988f7542 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:36.453574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc050810-c22e-4c9b-b7aa-355413b4727e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:36.453846 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce107478-08dc-44ac-b0a5-f713988f7542 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:36.454982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce107478-08dc-44ac-b0a5-f713988f7542 None admin] RBAC: Adding query filter params (interface=public, service_id=8cf738d98e1241c49e67ccee9ca547ae, region_id=RegionOne) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:36.455972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc050810-c22e-4c9b-b7aa-355413b4727e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:36.456133 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce107478-08dc-44ac-b0a5-f713988f7542 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:36.458508 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce107478-08dc-44ac-b0a5-f713988f7542 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:36.471507 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 440/892] 10.222.0.76 () {56 vars in 1251 bytes} [Thu May 26 17:51:36 2022] GET /identity/v3/endpoints?service_id=8cf738d98e1241c49e67ccee9ca547ae&interface=public®ion_id=RegionOne => generated 194 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:36.472600 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce107478-08dc-44ac-b0a5-f713988f7542 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:36.476831 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 453/893] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:36.483688 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc050810-c22e-4c9b-b7aa-355413b4727e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:36.603448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce107478-08dc-44ac-b0a5-f713988f7542 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:36.618963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc050810-c22e-4c9b-b7aa-355413b4727e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:36.641533 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:36.659366 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3be3b-5297-4f4d-b77c-a54b83aa240f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:36.661249 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3be3b-5297-4f4d-b77c-a54b83aa240f None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:36.661872 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:36.662226 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3be3b-5297-4f4d-b77c-a54b83aa240f None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:36.662743 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3be3b-5297-4f4d-b77c-a54b83aa240f None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:36.663779 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3be3b-5297-4f4d-b77c-a54b83aa240f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:36.665473 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3be3b-5297-4f4d-b77c-a54b83aa240f None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:36.678493 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80518bd7-7874-4ca9-92ac-ab6e2487196b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:36.678493 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80518bd7-7874-4ca9-92ac-ab6e2487196b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:36.678493 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80518bd7-7874-4ca9-92ac-ab6e2487196b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:36.678493 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80518bd7-7874-4ca9-92ac-ab6e2487196b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:36.678971 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 441/894] 10.222.0.76 () {56 vars in 1087 bytes} [Thu May 26 17:51:36 2022] GET /identity/v3/services => generated 1268 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 17:51:36.692452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80518bd7-7874-4ca9-92ac-ab6e2487196b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:36.694104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80518bd7-7874-4ca9-92ac-ab6e2487196b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:36.710508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-80518bd7-7874-4ca9-92ac-ab6e2487196b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:36.710508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:36.710508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:36.710508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:36.710508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:36.710508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:36.710508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:36.710508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:36.710508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:36.710508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:36.710508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:36.710508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:36.710508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:51:36.710508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:51:36.710508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:51:36.711460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:51:36.711460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:36.711460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:36.711460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:36.711460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:36.711460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:36.711460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:36.711460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:36.711460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:36.711460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:36.711460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:36.711460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:36.711460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:36.711460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:36.711460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:36.711460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:36.711460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:36.712577 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:51:36.712577 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:51:36.712577 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:51:36.712577 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:51:36.712577 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:51:36.712577 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:51:36.712577 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:36.712577 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:36.712577 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 454/895] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:36.719177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3be3b-5297-4f4d-b77c-a54b83aa240f None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:36.861112 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd4cb7-eb0e-4d6e-96d9-3522906174d7 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:36.861596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd4cb7-eb0e-4d6e-96d9-3522906174d7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:36.861980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd4cb7-eb0e-4d6e-96d9-3522906174d7 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:36.864007 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 455/896] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:36.867203 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3be3b-5297-4f4d-b77c-a54b83aa240f None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:36.874016 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae070f-3b1f-47f6-b5f9-1e24b6f7f283 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:36.874016 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae070f-3b1f-47f6-b5f9-1e24b6f7f283 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:36.874016 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae070f-3b1f-47f6-b5f9-1e24b6f7f283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:36.897102 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e2ae070f-3b1f-47f6-b5f9-1e24b6f7f283 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:36.898264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e2ae070f-3b1f-47f6-b5f9-1e24b6f7f283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:36.923220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ee37d-cca2-48d3-a7f4-15f2dd91ca9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:36.923942 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ee37d-cca2-48d3-a7f4-15f2dd91ca9f None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:36.924228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ee37d-cca2-48d3-a7f4-15f2dd91ca9f None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:36.924507 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ee37d-cca2-48d3-a7f4-15f2dd91ca9f None admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:36.925311 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ee37d-cca2-48d3-a7f4-15f2dd91ca9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:36.925635 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ee37d-cca2-48d3-a7f4-15f2dd91ca9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:36.927732 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ee37d-cca2-48d3-a7f4-15f2dd91ca9f None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:36.950217 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 442/897] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:51:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:36.953540 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ee37d-cca2-48d3-a7f4-15f2dd91ca9f None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:37.000998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 456/898] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:37.013770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:37.030952 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae070f-3b1f-47f6-b5f9-1e24b6f7f283 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:37.084437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ee37d-cca2-48d3-a7f4-15f2dd91ca9f None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:37.141946 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f5191-e883-4385-9afb-e173991802ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:37.144157 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f5191-e883-4385-9afb-e173991802ab None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:37.145249 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f5191-e883-4385-9afb-e173991802ab None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:37.145709 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f5191-e883-4385-9afb-e173991802ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:37.147650 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f5191-e883-4385-9afb-e173991802ab None admin] RBAC: Adding query filter params (role.id=04c312d9dc3249fe8fbff78befef65f9, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=3e06d1b0e1a9472db764f4be7e42fdd9) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:37.148243 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f5191-e883-4385-9afb-e173991802ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:37.150151 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f5191-e883-4385-9afb-e173991802ab None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:37.159782 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 443/899] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:51:36 2022] GET /identity/v3/role_assignments?user.id=3e06d1b0e1a9472db764f4be7e42fdd9&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=04c312d9dc3249fe8fbff78befef65f9 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:37.173380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae070f-3b1f-47f6-b5f9-1e24b6f7f283 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:37.197800 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:37.222420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c527c51-8cde-4893-8ecc-d65cd45be5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:37.223288 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c527c51-8cde-4893-8ecc-d65cd45be5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:37.223646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c527c51-8cde-4893-8ecc-d65cd45be5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:37.223987 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c527c51-8cde-4893-8ecc-d65cd45be5d1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:37.224675 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c527c51-8cde-4893-8ecc-d65cd45be5d1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:37.226368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c527c51-8cde-4893-8ecc-d65cd45be5d1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:37.242790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-9c527c51-8cde-4893-8ecc-d65cd45be5d1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 17:51:37.242790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:37.242790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:37.242790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:37.242790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:37.242790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:37.242790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:37.242790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:37.242790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 17:51:37.242790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:37.242790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:37.242790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:37.242790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:37.242790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:37.242790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:37.242790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:37.243818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:37.243818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:37.243818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:37.243818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:37.243818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:37.243818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:37.243818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:37.243818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:37.243818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:37.243818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:37.243818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:37.243818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:37.243818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:37.243818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:37.243818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:37.243818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:37.244660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:37.244660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:37.244660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:37.244660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:37.244660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:37.244660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:37.244660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:37.244660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:37.244660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:37.244660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:37.244660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:37.244660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:37.244660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:37.244660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 17:51:37.244660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:37.244660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 457/900] 10.222.0.76 () {56 vars in 1132 bytes} [Thu May 26 17:51:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:51:37.253098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f5191-e883-4385-9afb-e173991802ab None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:37.384320 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f5191-e883-4385-9afb-e173991802ab None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:37.400281 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:37.424005 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408dc70-1156-4e9c-9df4-c0afc232a92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:37.424689 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408dc70-1156-4e9c-9df4-c0afc232a92c None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:37.424968 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408dc70-1156-4e9c-9df4-c0afc232a92c None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:37.425246 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408dc70-1156-4e9c-9df4-c0afc232a92c None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:37.425965 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408dc70-1156-4e9c-9df4-c0afc232a92c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:37.426313 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408dc70-1156-4e9c-9df4-c0afc232a92c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:37.427702 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408dc70-1156-4e9c-9df4-c0afc232a92c None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:37.433435 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 444/901] 10.222.0.76 () {56 vars in 1126 bytes} [Thu May 26 17:51:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:37.439877 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c527c51-8cde-4893-8ecc-d65cd45be5d1 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:37.572534 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c527c51-8cde-4893-8ecc-d65cd45be5d1 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:37.617655 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59003770-6d87-4b81-99fa-b946f1932464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:37.618538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59003770-6d87-4b81-99fa-b946f1932464 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:37.618822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59003770-6d87-4b81-99fa-b946f1932464 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:37.619098 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59003770-6d87-4b81-99fa-b946f1932464 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:37.619899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59003770-6d87-4b81-99fa-b946f1932464 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:37.620328 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59003770-6d87-4b81-99fa-b946f1932464 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:37.622053 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59003770-6d87-4b81-99fa-b946f1932464 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:37.629833 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 458/902] 10.222.0.76 () {56 vars in 1126 bytes} [Thu May 26 17:51:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:37.635916 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408dc70-1156-4e9c-9df4-c0afc232a92c None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:37.747544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408dc70-1156-4e9c-9df4-c0afc232a92c None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:37.784241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04145b-7eec-4fb3-9b02-7714358ad6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:37.784923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04145b-7eec-4fb3-9b02-7714358ad6cd None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:37.785207 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04145b-7eec-4fb3-9b02-7714358ad6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:37.785484 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04145b-7eec-4fb3-9b02-7714358ad6cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:37.786576 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad04145b-7eec-4fb3-9b02-7714358ad6cd None admin] RBAC: Adding query filter params (interface=public, service_id=38a2017ef34a40208b3fe3cf814d8aae, region_id=RegionOne) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:37.786889 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad04145b-7eec-4fb3-9b02-7714358ad6cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:37.788270 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad04145b-7eec-4fb3-9b02-7714358ad6cd None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:37.795273 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 445/903] 10.222.0.76 () {56 vars in 1251 bytes} [Thu May 26 17:51:37 2022] GET /identity/v3/endpoints?service_id=38a2017ef34a40208b3fe3cf814d8aae&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:37.801446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59003770-6d87-4b81-99fa-b946f1932464 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:37.882871 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:37.921954 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59003770-6d87-4b81-99fa-b946f1932464 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:37.960991 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329798c-9fcc-4b8d-aee2-0a85279fb7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:37.961666 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329798c-9fcc-4b8d-aee2-0a85279fb7a6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:37.961944 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329798c-9fcc-4b8d-aee2-0a85279fb7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:37.962235 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329798c-9fcc-4b8d-aee2-0a85279fb7a6 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:37.962923 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329798c-9fcc-4b8d-aee2-0a85279fb7a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:37.964306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329798c-9fcc-4b8d-aee2-0a85279fb7a6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:37.971679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 459/904] 10.222.0.76 () {56 vars in 1087 bytes} [Thu May 26 17:51:37 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:51:38.715128 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455efcaf-531b-4c62-8ab6-b442f2a50631 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:38.716176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455efcaf-531b-4c62-8ab6-b442f2a50631 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:38.716705 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455efcaf-531b-4c62-8ab6-b442f2a50631 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:38.718578 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 446/905] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:38.724788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bc175-a4f4-4fd5-9d5a-0d2e105f5c2c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:38.725653 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bc175-a4f4-4fd5-9d5a-0d2e105f5c2c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:38.726135 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bc175-a4f4-4fd5-9d5a-0d2e105f5c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:38.744638 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4d2bc175-a4f4-4fd5-9d5a-0d2e105f5c2c None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:38.746291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4d2bc175-a4f4-4fd5-9d5a-0d2e105f5c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:38.824148 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 460/906] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:38.860479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455efcaf-531b-4c62-8ab6-b442f2a50631 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:38.929254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:38.990520 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455efcaf-531b-4c62-8ab6-b442f2a50631 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:39.025210 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c28e74-739c-417e-9894-5616efe80811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:39.026076 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c28e74-739c-417e-9894-5616efe80811 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:39.026393 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c28e74-739c-417e-9894-5616efe80811 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:39.026666 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c28e74-739c-417e-9894-5616efe80811 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:39.027328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c28e74-739c-417e-9894-5616efe80811 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:39.028914 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c28e74-739c-417e-9894-5616efe80811 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:39.043513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-02c28e74-739c-417e-9894-5616efe80811 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:51:39.043513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:39.043513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:39.043513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:39.043513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:39.043513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:39.043513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:39.043513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:39.043513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:51:39.043513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:39.043513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:39.043513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:39.043513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:39.043513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:39.043513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:39.043513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:39.043513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:39.045140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:39.045140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:39.045140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:39.045140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:39.045140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:39.045140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:39.045140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:39.045140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:39.045140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:39.045140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:39.045140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:39.045140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:39.045140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:39.045140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:39.045140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:39.045140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:39.046617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:39.046617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:39.046617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:39.046617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:39.046617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:39.046617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:39.046617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:39.046617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:39.046617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:39.046617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:39.046617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:39.046617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:39.046617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:51:39.046617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:39.046617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 447/907] 10.222.0.76 () {56 vars in 1126 bytes} [Thu May 26 17:51:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:39.058398 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2bc175-a4f4-4fd5-9d5a-0d2e105f5c2c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:39.124437 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:39.201649 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2bc175-a4f4-4fd5-9d5a-0d2e105f5c2c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:39.241098 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73212e6-a189-49a4-91b3-03c1995a48e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:39.241681 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73212e6-a189-49a4-91b3-03c1995a48e7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:39.241963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73212e6-a189-49a4-91b3-03c1995a48e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:39.242266 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73212e6-a189-49a4-91b3-03c1995a48e7 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:39.243006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73212e6-a189-49a4-91b3-03c1995a48e7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:39.243314 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73212e6-a189-49a4-91b3-03c1995a48e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:39.244823 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73212e6-a189-49a4-91b3-03c1995a48e7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:39.250904 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 461/908] 10.222.0.76 () {56 vars in 1122 bytes} [Thu May 26 17:51:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:39.260335 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c28e74-739c-417e-9894-5616efe80811 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:39.310843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a8e18-6b07-4657-afe8-747cd0bed744 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:39.311251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a8e18-6b07-4657-afe8-747cd0bed744 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:39.311401 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a8e18-6b07-4657-afe8-747cd0bed744 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:39.313316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 462/909] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:39.320236 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70842137-3222-437d-aa0b-1663392f1d25 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:39.320638 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70842137-3222-437d-aa0b-1663392f1d25 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:39.320778 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70842137-3222-437d-aa0b-1663392f1d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:39.342795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-70842137-3222-437d-aa0b-1663392f1d25 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:39.344011 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-70842137-3222-437d-aa0b-1663392f1d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:39.417005 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c28e74-739c-417e-9894-5616efe80811 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:39.425279 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:39.437709 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 463/910] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:51:39.440390 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6f125-7fdb-4783-b3c4-dfd27ef74a7f None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:39.440714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6f125-7fdb-4783-b3c4-dfd27ef74a7f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:39.441112 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6f125-7fdb-4783-b3c4-dfd27ef74a7f None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:39.446247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 464/911] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:39.449491 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2f902-961d-45b2-bf26-89456ad7b99b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:39.450182 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2f902-961d-45b2-bf26-89456ad7b99b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:39.451143 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2f902-961d-45b2-bf26-89456ad7b99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:39.470010 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f983e5-a377-4fff-82ef-51457a6dc89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:39.471339 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f983e5-a377-4fff-82ef-51457a6dc89d None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:39.471615 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f983e5-a377-4fff-82ef-51457a6dc89d None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:39.471997 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f983e5-a377-4fff-82ef-51457a6dc89d None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:39.472720 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f983e5-a377-4fff-82ef-51457a6dc89d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:39.473035 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f983e5-a377-4fff-82ef-51457a6dc89d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:39.474453 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f983e5-a377-4fff-82ef-51457a6dc89d None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:39.478039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-57a2f902-961d-45b2-bf26-89456ad7b99b None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:39.479628 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 448/912] 10.222.0.76 () {56 vars in 1122 bytes} [Thu May 26 17:51:39 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:39.480189 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-57a2f902-961d-45b2-bf26-89456ad7b99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:39.480528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f983e5-a377-4fff-82ef-51457a6dc89d None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:39.528997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:39.589162 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 465/913] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:39.622255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2f902-961d-45b2-bf26-89456ad7b99b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:39.665103 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f983e5-a377-4fff-82ef-51457a6dc89d None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:39.724981 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef491c-7b8f-4917-a171-21d1be3f6ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:39.726903 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef491c-7b8f-4917-a171-21d1be3f6ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:39.727927 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef491c-7b8f-4917-a171-21d1be3f6ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:39.728702 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef491c-7b8f-4917-a171-21d1be3f6ea2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:39.729598 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef491c-7b8f-4917-a171-21d1be3f6ea2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:39.731272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef491c-7b8f-4917-a171-21d1be3f6ea2 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:39.747381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-44ef491c-7b8f-4917-a171-21d1be3f6ea2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 17:51:39.747381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:39.747381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:39.747381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:39.747381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:39.747381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:39.747381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:39.747381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:39.747381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 17:51:39.747381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:39.747381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:39.747381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:39.747381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:39.747381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:39.747381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:39.747381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:39.747381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:39.748474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:39.748474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:39.748474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:39.748474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:39.748474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:39.748474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:39.748474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:39.748474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:39.748474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:39.748474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:39.748474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:39.748474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:39.748474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:39.748474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:39.748474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:39.748474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:39.749322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:39.749322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:39.749322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:39.749322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:39.749322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:39.749322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:39.749322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:39.749322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:39.749322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:39.749322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:39.749322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:39.749322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:39.749322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 17:51:39.749322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:39.750862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 449/914] 10.222.0.76 () {56 vars in 1114 bytes} [Thu May 26 17:51:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:39.758864 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef491c-7b8f-4917-a171-21d1be3f6ea2 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:39.796263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2f902-961d-45b2-bf26-89456ad7b99b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:39.818914 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:39.860873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4555d4a-76bf-4bfa-a4fa-7ce21de9ed70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:39.863229 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4555d4a-76bf-4bfa-a4fa-7ce21de9ed70 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:39.864075 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4555d4a-76bf-4bfa-a4fa-7ce21de9ed70 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:39.865036 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4555d4a-76bf-4bfa-a4fa-7ce21de9ed70 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:39.881964 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4555d4a-76bf-4bfa-a4fa-7ce21de9ed70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:51:39.892907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-d4555d4a-76bf-4bfa-a4fa-7ce21de9ed70 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:51:39.892907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:39.892907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:39.892907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:39.892907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:39.892907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:39.892907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:39.892907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:39.892907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:39.892907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:39.892907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:39.892907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:39.892907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:51:39.892907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:51:39.892907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:51:39.892907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:51:39.894247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:51:39.894247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:51:39.894247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:51:39.894247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:51:39.894247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:39.894247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:39.894247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:39.894247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:39.894247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:39.894247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:39.894247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:39.894247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:39.894247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:39.894247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:39.894247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:39.894247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:39.895727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:39.895727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:39.895727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:39.895727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:39.895727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:51:39.895727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:51:39.895727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:51:39.895727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:51:39.895727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:51:39.895727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:51:39.895727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:51:39.895727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:39.898300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 466/915] 10.222.0.76 () {56 vars in 1120 bytes} [Thu May 26 17:51:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:39.901823 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4555d4a-76bf-4bfa-a4fa-7ce21de9ed70 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:39.933969 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:39.934519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef491c-7b8f-4917-a171-21d1be3f6ea2 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:39.983558 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587648f-07ea-405a-b263-c74a73dbe1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:39.984869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587648f-07ea-405a-b263-c74a73dbe1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:39.985339 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587648f-07ea-405a-b263-c74a73dbe1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:39.985798 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587648f-07ea-405a-b263-c74a73dbe1c9 None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:39.987563 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587648f-07ea-405a-b263-c74a73dbe1c9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:39.988064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587648f-07ea-405a-b263-c74a73dbe1c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:39.990114 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587648f-07ea-405a-b263-c74a73dbe1c9 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:39.997126 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 450/916] 10.222.0.76 () {56 vars in 1114 bytes} [Thu May 26 17:51:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:40.006472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587648f-07ea-405a-b263-c74a73dbe1c9 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:40.061506 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4555d4a-76bf-4bfa-a4fa-7ce21de9ed70 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:40.109229 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac49f4-590e-44d2-9df7-a3299de2714a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:40.110331 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac49f4-590e-44d2-9df7-a3299de2714a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:40.111509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac49f4-590e-44d2-9df7-a3299de2714a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:40.111864 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac49f4-590e-44d2-9df7-a3299de2714a None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:40.113039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac49f4-590e-44d2-9df7-a3299de2714a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:40.113731 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac49f4-590e-44d2-9df7-a3299de2714a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:40.115462 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac49f4-590e-44d2-9df7-a3299de2714a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:40.135436 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:40.142336 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587648f-07ea-405a-b263-c74a73dbe1c9 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:40.148025 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 467/917] 10.222.0.76 () {56 vars in 1115 bytes} [Thu May 26 17:51:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:40.149465 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac49f4-590e-44d2-9df7-a3299de2714a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:40.188554 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec138a-f601-4291-b051-0d54032e8ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:40.190330 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec138a-f601-4291-b051-0d54032e8ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:40.190823 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec138a-f601-4291-b051-0d54032e8ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:40.191330 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec138a-f601-4291-b051-0d54032e8ab8 None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:40.192265 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec138a-f601-4291-b051-0d54032e8ab8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:40.192765 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec138a-f601-4291-b051-0d54032e8ab8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:40.194496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec138a-f601-4291-b051-0d54032e8ab8 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:40.205246 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 451/918] 10.222.0.76 () {56 vars in 1114 bytes} [Thu May 26 17:51:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:40.206361 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec138a-f601-4291-b051-0d54032e8ab8 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:40.253750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:40.276161 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac49f4-590e-44d2-9df7-a3299de2714a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:40.326026 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec138a-f601-4291-b051-0d54032e8ab8 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:40.326988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29debf-4ab9-4201-9165-ff45ac394ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:40.328350 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29debf-4ab9-4201-9165-ff45ac394ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:40.329710 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29debf-4ab9-4201-9165-ff45ac394ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:40.330456 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29debf-4ab9-4201-9165-ff45ac394ba4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:40.344696 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac29debf-4ab9-4201-9165-ff45ac394ba4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:40.346761 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac29debf-4ab9-4201-9165-ff45ac394ba4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:40.363849 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ce5c2-0783-4e73-b515-bb02509e91dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:40.364894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-ac29debf-4ab9-4201-9165-ff45ac394ba4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:40.364894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:40.364894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:40.364894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:40.364894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:40.364894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:40.364894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:40.364894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:40.364894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:40.364894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:40.364894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:40.364894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:40.364894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:40.364894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:40.364894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:40.365766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:40.365766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:40.365766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:40.365766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:40.365766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:40.365766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:40.365766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:40.365766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:40.365766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:40.365766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:40.365766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:40.365766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:40.365766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:40.365766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:40.365766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:40.365766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:40.365766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:40.366693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:40.366693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:40.366693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:40.366693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:40.366693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 468/919] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:51:40.367463 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ce5c2-0783-4e73-b515-bb02509e91dd None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:40.367565 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5edad-6874-498e-8bdb-fd15f76ae5a0 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:40.367801 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5edad-6874-498e-8bdb-fd15f76ae5a0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:40.368067 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5edad-6874-498e-8bdb-fd15f76ae5a0 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:40.368477 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ce5c2-0783-4e73-b515-bb02509e91dd None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:40.368942 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ce5c2-0783-4e73-b515-bb02509e91dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:40.369335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 469/920] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:40.370097 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ce5c2-0783-4e73-b515-bb02509e91dd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:40.371696 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ce5c2-0783-4e73-b515-bb02509e91dd None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:40.372021 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5edad-6874-498e-8bdb-fd15f76ae5a0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:40.397799 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 452/921] 10.222.0.76 () {60 vars in 1144 bytes} [Thu May 26 17:51:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:51:40.397799 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e4e20-b94c-480a-9477-f75152f1bca9 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:40.397799 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e4e20-b94c-480a-9477-f75152f1bca9 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:40.397799 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e4e20-b94c-480a-9477-f75152f1bca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:40.412052 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-999e4e20-b94c-480a-9477-f75152f1bca9 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:40.413290 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-999e4e20-b94c-480a-9477-f75152f1bca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:40.432669 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:40.507487 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 453/922] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:40.531243 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e4e20-b94c-480a-9477-f75152f1bca9 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:40.539688 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5edad-6874-498e-8bdb-fd15f76ae5a0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:40.593873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbaf26e-63d3-4da0-9a5b-85c08d92bb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:40.595265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbaf26e-63d3-4da0-9a5b-85c08d92bb13 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:40.596476 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbaf26e-63d3-4da0-9a5b-85c08d92bb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:40.597274 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbaf26e-63d3-4da0-9a5b-85c08d92bb13 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:40.599973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbaf26e-63d3-4da0-9a5b-85c08d92bb13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:40.600294 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbaf26e-63d3-4da0-9a5b-85c08d92bb13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:40.601722 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbaf26e-63d3-4da0-9a5b-85c08d92bb13 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:40.609396 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:40.617056 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 470/923] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:40.622673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbaf26e-63d3-4da0-9a5b-85c08d92bb13 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:40.663558 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e4e20-b94c-480a-9477-f75152f1bca9 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:40.707823 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279c00f-e27e-44d6-a182-61458fb9ba6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:40.708656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279c00f-e27e-44d6-a182-61458fb9ba6e None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:40.708943 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279c00f-e27e-44d6-a182-61458fb9ba6e None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:40.709225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279c00f-e27e-44d6-a182-61458fb9ba6e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:40.709888 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279c00f-e27e-44d6-a182-61458fb9ba6e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:40.711662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279c00f-e27e-44d6-a182-61458fb9ba6e None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:40.727109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-e279c00f-e27e-44d6-a182-61458fb9ba6e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 17:51:40.727109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:40.727109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:40.727109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:40.727109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:40.727109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:40.727109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:40.727109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:40.727109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 17:51:40.727109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:40.727109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:40.727109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:40.727109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:40.727109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:40.727109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:40.727109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:40.728528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:40.728528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:40.728528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:40.728528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:40.728528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:40.728528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:40.728528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:40.728528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:40.728528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:40.728528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:40.728528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:40.728528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:40.728528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:40.728528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:40.728528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:40.728528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:40.729362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:40.729362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:40.729362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:40.729362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:40.729362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:40.729362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:40.729362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:40.729362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:40.729362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:40.729362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:40.729362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:40.729362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:40.729362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:40.729362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 17:51:40.729362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:40.729362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 454/924] 10.222.0.76 () {56 vars in 1132 bytes} [Thu May 26 17:51:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:51:40.734811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279c00f-e27e-44d6-a182-61458fb9ba6e None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:40.756022 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:40.756253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbaf26e-63d3-4da0-9a5b-85c08d92bb13 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:40.802868 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7950e9c-81ca-440a-ab04-f44f00ac7af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:40.803995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7950e9c-81ca-440a-ab04-f44f00ac7af1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:40.804282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7950e9c-81ca-440a-ab04-f44f00ac7af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:40.804562 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7950e9c-81ca-440a-ab04-f44f00ac7af1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:40.816088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7950e9c-81ca-440a-ab04-f44f00ac7af1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:40.818018 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7950e9c-81ca-440a-ab04-f44f00ac7af1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:40.834795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-a7950e9c-81ca-440a-ab04-f44f00ac7af1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:51:40.834795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:40.834795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:40.834795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:40.834795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:40.834795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:40.834795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:40.834795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:40.834795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:40.834795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:40.834795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:40.834795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:40.834795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:51:40.834795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:51:40.834795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:51:40.834795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:51:40.835814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:40.835814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:40.835814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:51:40.835814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:40.835814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:51:40.835814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:40.835814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:40.835814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:40.835814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:40.835814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:40.835814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:40.835814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:40.835814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:40.835814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:40.835814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:40.835814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:40.835814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:40.835814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:40.836755 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:40.836755 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:40.836755 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:51:40.836755 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:51:40.836755 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:51:40.836755 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:51:40.836755 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:51:40.836755 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:51:40.836755 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:51:40.836755 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:40.836755 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 471/925] 10.222.0.76 () {56 vars in 1117 bytes} [Thu May 26 17:51:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:40.847762 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7950e9c-81ca-440a-ab04-f44f00ac7af1 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:40.883842 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279c00f-e27e-44d6-a182-61458fb9ba6e None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:40.922064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93bbb6-2aad-4ec9-90fb-5763b1d836eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:40.922662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:40.923119 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93bbb6-2aad-4ec9-90fb-5763b1d836eb None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:40.923282 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93bbb6-2aad-4ec9-90fb-5763b1d836eb None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:40.923606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93bbb6-2aad-4ec9-90fb-5763b1d836eb None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:40.924457 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93bbb6-2aad-4ec9-90fb-5763b1d836eb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:40.924856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93bbb6-2aad-4ec9-90fb-5763b1d836eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:40.926373 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93bbb6-2aad-4ec9-90fb-5763b1d836eb None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:40.932907 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 455/926] 10.222.0.76 () {56 vars in 1126 bytes} [Thu May 26 17:51:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:40.940401 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93bbb6-2aad-4ec9-90fb-5763b1d836eb None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:40.986028 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7950e9c-81ca-440a-ab04-f44f00ac7af1 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:41.027354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:41.041636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f9b68-71f8-43f8-9c77-e85319a398fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:41.043309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f9b68-71f8-43f8-9c77-e85319a398fb None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:41.043818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f9b68-71f8-43f8-9c77-e85319a398fb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:41.044537 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f9b68-71f8-43f8-9c77-e85319a398fb None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:41.045579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f9b68-71f8-43f8-9c77-e85319a398fb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:41.046334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f9b68-71f8-43f8-9c77-e85319a398fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:41.048536 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f9b68-71f8-43f8-9c77-e85319a398fb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:41.071073 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93bbb6-2aad-4ec9-90fb-5763b1d836eb None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:41.074490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 472/927] 10.222.0.76 () {56 vars in 1149 bytes} [Thu May 26 17:51:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:41.081288 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f9b68-71f8-43f8-9c77-e85319a398fb None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:41.115643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16338c35-f9b2-4fab-9e06-f2aa22a70108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:41.116233 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16338c35-f9b2-4fab-9e06-f2aa22a70108 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:41.116576 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16338c35-f9b2-4fab-9e06-f2aa22a70108 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:41.116862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16338c35-f9b2-4fab-9e06-f2aa22a70108 None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:41.117638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16338c35-f9b2-4fab-9e06-f2aa22a70108 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:41.117987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16338c35-f9b2-4fab-9e06-f2aa22a70108 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:41.119490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16338c35-f9b2-4fab-9e06-f2aa22a70108 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:41.135917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 456/928] 10.222.0.76 () {56 vars in 1126 bytes} [Thu May 26 17:51:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:41.144048 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16338c35-f9b2-4fab-9e06-f2aa22a70108 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:41.229879 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f9b68-71f8-43f8-9c77-e85319a398fb None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:41.256025 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:41.280426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402e281-e88d-440c-9f81-e67f06b4feda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:41.282148 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402e281-e88d-440c-9f81-e67f06b4feda None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:41.282680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402e281-e88d-440c-9f81-e67f06b4feda None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:41.283762 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402e281-e88d-440c-9f81-e67f06b4feda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:41.298205 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402e281-e88d-440c-9f81-e67f06b4feda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:41.301683 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402e281-e88d-440c-9f81-e67f06b4feda None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:41.309872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16338c35-f9b2-4fab-9e06-f2aa22a70108 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:41.325832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c402e281-e88d-440c-9f81-e67f06b4feda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:41.325832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:41.325832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:41.325832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:41.325832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:41.325832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:41.325832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:41.325832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:41.325832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:41.325832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:41.325832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:41.325832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:41.325832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:41.325832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:41.325832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:41.326851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:41.326851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:41.326851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:41.326851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:41.326851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:41.326851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:41.326851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:41.326851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:41.326851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:41.326851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:41.326851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:41.326851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:41.326851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:41.326851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:41.326851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:41.326851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:41.326851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:41.327783 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:41.327783 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:41.327783 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:41.327783 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:41.330640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 473/929] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:41.336673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402e281-e88d-440c-9f81-e67f06b4feda None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:41.377013 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018f077-4276-4bac-b91b-212e95612592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:41.377932 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018f077-4276-4bac-b91b-212e95612592 None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:41.378611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018f077-4276-4bac-b91b-212e95612592 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:41.378902 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018f077-4276-4bac-b91b-212e95612592 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:41.380164 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018f077-4276-4bac-b91b-212e95612592 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:41.383165 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018f077-4276-4bac-b91b-212e95612592 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:41.389638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:41.418415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 457/930] 10.222.0.76 () {60 vars in 1144 bytes} [Thu May 26 17:51:41 2022] POST /identity/v3/endpoints => generated 347 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:51:41.475314 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402e281-e88d-440c-9f81-e67f06b4feda None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:41.523601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ec33b-95c9-42e3-9792-3a6cf74f4da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:41.524287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ec33b-95c9-42e3-9792-3a6cf74f4da6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:41.524630 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ec33b-95c9-42e3-9792-3a6cf74f4da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:41.524913 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ec33b-95c9-42e3-9792-3a6cf74f4da6 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:41.525652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ec33b-95c9-42e3-9792-3a6cf74f4da6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:41.525967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ec33b-95c9-42e3-9792-3a6cf74f4da6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:41.527488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ec33b-95c9-42e3-9792-3a6cf74f4da6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:41.542529 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 474/931] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:41.548833 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018f077-4276-4bac-b91b-212e95612592 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:41.721844 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018f077-4276-4bac-b91b-212e95612592 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:41.769483 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26868717-063a-449e-8f27-444c0d7af8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:41.770367 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26868717-063a-449e-8f27-444c0d7af8ca None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:41.770651 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26868717-063a-449e-8f27-444c0d7af8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:41.770933 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26868717-063a-449e-8f27-444c0d7af8ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:41.779577 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26868717-063a-449e-8f27-444c0d7af8ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:41.781113 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26868717-063a-449e-8f27-444c0d7af8ca None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:41.799574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-26868717-063a-449e-8f27-444c0d7af8ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:41.799574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:41.799574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:41.799574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:41.799574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:41.799574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:41.799574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:41.799574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:41.799574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:41.799574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:41.799574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:41.799574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:41.799574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:51:41.799574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:51:41.799574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:51:41.800569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:51:41.800569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:41.800569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:41.800569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:41.800569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:41.800569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:41.800569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:41.800569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:41.800569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:41.800569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:41.800569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:41.800569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:41.800569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:41.800569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:41.800569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:41.800569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:41.800569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:41.801740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:51:41.801740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:51:41.801740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:51:41.801740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:51:41.801740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:51:41.801740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:51:41.801740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:41.801740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:41.801740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 458/932] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:41.809675 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ec33b-95c9-42e3-9792-3a6cf74f4da6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:41.839890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:41.929802 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ec33b-95c9-42e3-9792-3a6cf74f4da6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:41.969151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10a878-f405-4777-b804-a7d264f46ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:41.970223 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10a878-f405-4777-b804-a7d264f46ddb None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:41.970894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10a878-f405-4777-b804-a7d264f46ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:41.971374 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10a878-f405-4777-b804-a7d264f46ddb None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:41.972564 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10a878-f405-4777-b804-a7d264f46ddb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:41.973104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10a878-f405-4777-b804-a7d264f46ddb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:41.973455 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881921a-8a80-421c-a4e0-fc650143ef27 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:41.973750 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881921a-8a80-421c-a4e0-fc650143ef27 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:41.974032 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881921a-8a80-421c-a4e0-fc650143ef27 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:41.975408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10a878-f405-4777-b804-a7d264f46ddb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:41.975731 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 459/933] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:51:41.981214 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b80573-0209-4679-a27e-a4bb702a9428 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:41.981506 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b80573-0209-4679-a27e-a4bb702a9428 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:41.981791 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b80573-0209-4679-a27e-a4bb702a9428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:41.995692 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 475/934] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:51:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:41.997285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10a878-f405-4777-b804-a7d264f46ddb None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:41.999766 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-75b80573-0209-4679-a27e-a4bb702a9428 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:42.001334 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-75b80573-0209-4679-a27e-a4bb702a9428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:42.063222 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:42.091125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 460/935] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:51:42.106865 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:42.114068 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b80573-0209-4679-a27e-a4bb702a9428 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:42.118705 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10a878-f405-4777-b804-a7d264f46ddb None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:42.169362 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607b625-1baa-41c9-a285-3a0cf25f2f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:42.170893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607b625-1baa-41c9-a285-3a0cf25f2f2c None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:42.171532 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607b625-1baa-41c9-a285-3a0cf25f2f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:42.172316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607b625-1baa-41c9-a285-3a0cf25f2f2c None admin] PATH_INFO: `/v3/projects/ca1d09f491d944f99425a641561b03c3/users/3e06d1b0e1a9472db764f4be7e42fdd9/roles/04c312d9dc3249fe8fbff78befef65f9` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:42.176187 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607b625-1baa-41c9-a285-3a0cf25f2f2c None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1d09f491d944f99425a641561b03c3, user_id=3e06d1b0e1a9472db764f4be7e42fdd9, role_id=04c312d9dc3249fe8fbff78befef65f9)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:42.178170 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607b625-1baa-41c9-a285-3a0cf25f2f2c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:42.206889 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 476/936] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:51:41 2022] PUT /identity/v3/projects/ca1d09f491d944f99425a641561b03c3/users/3e06d1b0e1a9472db764f4be7e42fdd9/roles/04c312d9dc3249fe8fbff78befef65f9 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 17:51:42.239121 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b80573-0209-4679-a27e-a4bb702a9428 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:42.282574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c41da3-be1e-4072-abe4-928aa008d41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:42.283590 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c41da3-be1e-4072-abe4-928aa008d41a None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:42.283881 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c41da3-be1e-4072-abe4-928aa008d41a None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:42.284159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c41da3-be1e-4072-abe4-928aa008d41a None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:42.284904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c41da3-be1e-4072-abe4-928aa008d41a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:42.294314 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c41da3-be1e-4072-abe4-928aa008d41a None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:42.309287 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 461/937] 10.222.0.76 () {60 vars in 1141 bytes} [Thu May 26 17:51:42 2022] POST /identity/v3/services => generated 241 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:51:42.909820 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f4941-0ef5-4c7c-957d-061bfb739067 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:42.910378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f4941-0ef5-4c7c-957d-061bfb739067 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:42.910378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f4941-0ef5-4c7c-957d-061bfb739067 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:42.912311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 477/938] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:42.917842 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6089f8-ac22-45bd-aee1-f2150c745936 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:42.918330 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6089f8-ac22-45bd-aee1-f2150c745936 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:42.918518 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6089f8-ac22-45bd-aee1-f2150c745936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:42.936788 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-4d6089f8-ac22-45bd-aee1-f2150c745936 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:42.938140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-4d6089f8-ac22-45bd-aee1-f2150c745936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:42.979404 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:43.041107 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 462/939] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:43.064113 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f4941-0ef5-4c7c-957d-061bfb739067 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:43.198053 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f4941-0ef5-4c7c-957d-061bfb739067 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:43.236078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad71b0-f98e-4b9a-a30f-ed0c5456770a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:43.237268 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad71b0-f98e-4b9a-a30f-ed0c5456770a None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:43.237750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad71b0-f98e-4b9a-a30f-ed0c5456770a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:43.238248 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad71b0-f98e-4b9a-a30f-ed0c5456770a None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:43.239197 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad71b0-f98e-4b9a-a30f-ed0c5456770a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:43.240919 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad71b0-f98e-4b9a-a30f-ed0c5456770a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:43.242724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: WARNING keystone.server.flask.common [None req-4aad71b0-f98e-4b9a-a30f-ed0c5456770a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 26 17:51:43.258738 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:43.262283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.sql.core [None req-4aad71b0-f98e-4b9a-a30f-ed0c5456770a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 17:51:43.262283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 26 17:51:43.262283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [parameters: {'id': 'cd237a15c7d84b87a054f6c5c2a0d381', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 26 17:51:43.262283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80261) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 17:51:43.282724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-4aad71b0-f98e-4b9a-a30f-ed0c5456770a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cd237a15c7d84b87a054f6c5c2a0d381.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cd237a15c7d84b87a054f6c5c2a0d381. May 26 17:51:43.282724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:43.282724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 17:51:43.282724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 17:51:43.282724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 17:51:43.282724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 17:51:43.282724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 17:51:43.282724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application result = self._query(query) May 26 17:51:43.282724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 17:51:43.282724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application conn.query(q) May 26 17:51:43.282724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 17:51:43.282724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 17:51:43.282724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 17:51:43.284013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application result.read() May 26 17:51:43.284013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 17:51:43.284013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 17:51:43.284013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 17:51:43.284013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 17:51:43.284013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 17:51:43.284013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 17:51:43.284013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 17:51:43.284013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 17:51:43.284013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 17:51:43.284013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:43.284013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 17:51:43.284013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:43.284013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:43.284013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 17:51:43.284013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 17:51:43.284013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 26 17:51:43.284013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return project_ref.to_dict() May 26 17:51:43.284978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 17:51:43.284978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application next(self.gen) May 26 17:51:43.284978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 17:51:43.284978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application yield resource May 26 17:51:43.284978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 17:51:43.284978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application next(self.gen) May 26 17:51:43.284978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 17:51:43.284978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.session.rollback() May 26 17:51:43.284978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 17:51:43.284978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.force_reraise() May 26 17:51:43.284978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 17:51:43.284978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise self.value May 26 17:51:43.284978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 17:51:43.284978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 17:51:43.284978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 17:51:43.284978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application session.commit() May 26 17:51:43.284978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 17:51:43.286147 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 17:51:43.286147 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 17:51:43.286147 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._prepare_impl() May 26 17:51:43.286147 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 17:51:43.286147 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.session.flush() May 26 17:51:43.286147 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 17:51:43.286147 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._flush(objects) May 26 17:51:43.286147 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 17:51:43.286147 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 17:51:43.286147 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 17:51:43.286147 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application compat.raise_( May 26 17:51:43.286147 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 17:51:43.286147 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception May 26 17:51:43.286147 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 17:51:43.286147 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application flush_context.execute() May 26 17:51:43.286147 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 17:51:43.286147 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rec.execute(self) May 26 17:51:43.287099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 17:51:43.287099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 17:51:43.287099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 17:51:43.287099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 17:51:43.287099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 26 17:51:43.287099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application c = connection._execute_20( May 26 17:51:43.287099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 17:51:43.287099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 17:51:43.287099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 17:51:43.287099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 17:51:43.287099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 17:51:43.287099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 17:51:43.287099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 17:51:43.287099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 17:51:43.287099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 17:51:43.287899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 17:51:43.287899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 17:51:43.287899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception May 26 17:51:43.287899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 17:51:43.287899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 17:51:43.287899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 17:51:43.287899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 17:51:43.287899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 17:51:43.287899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application result = self._query(query) May 26 17:51:43.287899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 17:51:43.287899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application conn.query(q) May 26 17:51:43.287899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 17:51:43.287899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 17:51:43.287899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 17:51:43.287899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application result.read() May 26 17:51:43.287899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 17:51:43.287899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 17:51:43.288750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 17:51:43.288750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 17:51:43.288750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 17:51:43.288750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 17:51:43.288750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 17:51:43.288750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 17:51:43.288750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 17:51:43.288750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 26 17:51:43.288750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [parameters: {'id': 'cd237a15c7d84b87a054f6c5c2a0d381', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 26 17:51:43.288750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 17:51:43.288750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:43.288750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:43.288750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:43.288750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:43.288750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 26 17:51:43.291414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 26 17:51:43.291414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 17:51:43.291414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 17:51:43.291414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 26 17:51:43.291414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:43.291414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:43.291414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:43.291414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:43.291414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:43.291414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:43.291414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:43.291414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:43.291414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:43.291414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:43.291414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:43.291414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:43.291414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:43.293567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:43.293567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 26 17:51:43.293567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 26 17:51:43.293567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:43.293567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:43.293567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 26 17:51:43.293567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.Conflict( May 26 17:51:43.293567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cd237a15c7d84b87a054f6c5c2a0d381. May 26 17:51:43.293567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:43.296282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 478/940] 10.222.0.76 () {60 vars in 1140 bytes} [Thu May 26 17:51:43 2022] POST /identity/v3/projects => generated 262 bytes in 233 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:43.306313 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6089f8-ac22-45bd-aee1-f2150c745936 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:43.419569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6089f8-ac22-45bd-aee1-f2150c745936 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:43.459222 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f4c4a-8521-47a1-9fe6-7363b2eaae89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:43.460270 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f4c4a-8521-47a1-9fe6-7363b2eaae89 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:43.460764 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f4c4a-8521-47a1-9fe6-7363b2eaae89 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:43.461205 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f4c4a-8521-47a1-9fe6-7363b2eaae89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:43.468995 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f4c4a-8521-47a1-9fe6-7363b2eaae89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:43.471693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f4c4a-8521-47a1-9fe6-7363b2eaae89 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:43.487809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-f55f4c4a-8521-47a1-9fe6-7363b2eaae89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:43.487809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:43.487809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:43.487809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:43.487809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:43.487809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:43.487809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:43.487809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:43.487809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:43.487809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:43.487809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:43.487809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:43.487809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:51:43.487809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:51:43.487809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:51:43.488913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:51:43.488913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:43.488913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:43.488913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:43.488913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:43.488913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:43.488913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:43.488913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:43.488913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:43.488913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:43.488913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:43.488913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:43.488913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:43.488913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:43.488913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:43.488913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:43.488913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:43.489928 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:51:43.489928 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:51:43.489928 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:51:43.489928 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:51:43.489928 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:51:43.489928 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:51:43.489928 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:43.489928 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:43.492080 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 463/941] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:43.498449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad71b0-f98e-4b9a-a30f-ed0c5456770a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:43.626478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad71b0-f98e-4b9a-a30f-ed0c5456770a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:43.667056 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af4387-dddc-47da-836c-a148900cdd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:43.668019 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af4387-dddc-47da-836c-a148900cdd1c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:43.668512 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af4387-dddc-47da-836c-a148900cdd1c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:43.668935 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af4387-dddc-47da-836c-a148900cdd1c None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:43.670078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af4387-dddc-47da-836c-a148900cdd1c None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:43.670566 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af4387-dddc-47da-836c-a148900cdd1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:43.672401 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af4387-dddc-47da-836c-a148900cdd1c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:43.695124 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 479/942] 10.222.0.76 () {56 vars in 1112 bytes} [Thu May 26 17:51:43 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:44.025875 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf62780-71bc-472c-8f67-8bf58c1db4fa None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:44.026475 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf62780-71bc-472c-8f67-8bf58c1db4fa None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:44.026475 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf62780-71bc-472c-8f67-8bf58c1db4fa None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:44.028099 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 464/943] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:44.033696 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941632a1-006b-44fb-9f27-21bef8655f83 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:44.034183 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941632a1-006b-44fb-9f27-21bef8655f83 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:44.034444 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941632a1-006b-44fb-9f27-21bef8655f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:44.053100 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-941632a1-006b-44fb-9f27-21bef8655f83 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:44.054707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-941632a1-006b-44fb-9f27-21bef8655f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:44.118030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:44.134352 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 480/944] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:44.157419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf62780-71bc-472c-8f67-8bf58c1db4fa None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:44.193431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:44.301385 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf62780-71bc-472c-8f67-8bf58c1db4fa None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:44.338671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46cbc1-3e1e-4fc4-b866-0bada7dcecb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:44.339481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46cbc1-3e1e-4fc4-b866-0bada7dcecb8 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:44.339771 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46cbc1-3e1e-4fc4-b866-0bada7dcecb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:44.340054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46cbc1-3e1e-4fc4-b866-0bada7dcecb8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:44.340717 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46cbc1-3e1e-4fc4-b866-0bada7dcecb8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:44.342108 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46cbc1-3e1e-4fc4-b866-0bada7dcecb8 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:44.355302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-3e46cbc1-3e1e-4fc4-b866-0bada7dcecb8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 17:51:44.355302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:44.355302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:44.355302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:44.355302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:44.355302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:44.355302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:44.355302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:44.355302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 17:51:44.355302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:44.355302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:44.355302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:44.355302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:44.355302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:44.355302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:44.355302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:44.355302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:44.356371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:44.356371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:44.356371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:44.356371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:44.356371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:44.356371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:44.356371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:44.356371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:44.356371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:44.356371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:44.356371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:44.356371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:44.356371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:44.356371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:44.356371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:44.356371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:44.357310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:44.357310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:44.357310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:44.357310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:44.357310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:44.357310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:44.357310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:44.357310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:44.357310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:44.357310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:44.357310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:44.357310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:44.357310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 17:51:44.357310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:44.357310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 465/945] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:44.366285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941632a1-006b-44fb-9f27-21bef8655f83 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:44.544164 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941632a1-006b-44fb-9f27-21bef8655f83 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:44.601272 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2427fef-ca3b-4b53-a4cc-99d4795b8c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:44.602244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2427fef-ca3b-4b53-a4cc-99d4795b8c41 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:44.602699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2427fef-ca3b-4b53-a4cc-99d4795b8c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:44.603154 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2427fef-ca3b-4b53-a4cc-99d4795b8c41 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:44.604324 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2427fef-ca3b-4b53-a4cc-99d4795b8c41 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:44.604813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2427fef-ca3b-4b53-a4cc-99d4795b8c41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:44.607123 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2427fef-ca3b-4b53-a4cc-99d4795b8c41 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:44.617229 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 481/946] 10.222.0.76 () {56 vars in 1112 bytes} [Thu May 26 17:51:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:44.625563 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46cbc1-3e1e-4fc4-b866-0bada7dcecb8 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:44.688823 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6e5fa-6442-4686-ae7a-dd14cec13e15 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:44.689376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6e5fa-6442-4686-ae7a-dd14cec13e15 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:44.689844 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6e5fa-6442-4686-ae7a-dd14cec13e15 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:44.692787 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 482/947] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:44.700049 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38378d28-6f3d-4f3d-aea6-b05d9c08079e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:44.700508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38378d28-6f3d-4f3d-aea6-b05d9c08079e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:44.700933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38378d28-6f3d-4f3d-aea6-b05d9c08079e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:44.721193 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-38378d28-6f3d-4f3d-aea6-b05d9c08079e None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:44.722677 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-38378d28-6f3d-4f3d-aea6-b05d9c08079e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:44.765679 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:44.783250 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:44.786092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46cbc1-3e1e-4fc4-b866-0bada7dcecb8 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:44.815486 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 483/948] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:44.837520 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac1f50-2b1c-44ce-afc9-369d1b4fc4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:44.838356 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac1f50-2b1c-44ce-afc9-369d1b4fc4ac None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:44.838831 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac1f50-2b1c-44ce-afc9-369d1b4fc4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:44.839255 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac1f50-2b1c-44ce-afc9-369d1b4fc4ac None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:44.841314 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38378d28-6f3d-4f3d-aea6-b05d9c08079e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:44.842322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac1f50-2b1c-44ce-afc9-369d1b4fc4ac None admin] RBAC: Adding query filter params (type=compute) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:44.842924 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac1f50-2b1c-44ce-afc9-369d1b4fc4ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:44.845001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac1f50-2b1c-44ce-afc9-369d1b4fc4ac None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:44.858175 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 466/949] 10.222.0.76 () {56 vars in 1112 bytes} [Thu May 26 17:51:44 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:44.966121 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f45ae0-4a69-4172-ab80-75f4a0e5d40c None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:44.966506 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f45ae0-4a69-4172-ab80-75f4a0e5d40c None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:44.966783 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f45ae0-4a69-4172-ab80-75f4a0e5d40c None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:44.973836 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 467/950] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:51:44.975964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283c7ab-dc60-49b2-bc0b-6b75f2c0f0fb None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:44.976254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283c7ab-dc60-49b2-bc0b-6b75f2c0f0fb None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:44.976533 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283c7ab-dc60-49b2-bc0b-6b75f2c0f0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:44.996966 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-1283c7ab-dc60-49b2-bc0b-6b75f2c0f0fb None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:44.998343 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-1283c7ab-dc60-49b2-bc0b-6b75f2c0f0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:45.008594 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38378d28-6f3d-4f3d-aea6-b05d9c08079e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:45.051340 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a666c7d-72e5-49e3-a396-8ff6c4ced98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:45.053249 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a666c7d-72e5-49e3-a396-8ff6c4ced98b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:45.053812 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a666c7d-72e5-49e3-a396-8ff6c4ced98b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:45.054328 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a666c7d-72e5-49e3-a396-8ff6c4ced98b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:45.055282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a666c7d-72e5-49e3-a396-8ff6c4ced98b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:45.056997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a666c7d-72e5-49e3-a396-8ff6c4ced98b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:45.072717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-9a666c7d-72e5-49e3-a396-8ff6c4ced98b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:51:45.072717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:45.072717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:45.072717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:45.072717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:45.072717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:45.072717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:45.072717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:45.072717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:51:45.072717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:45.072717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:45.072717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:45.072717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:45.072717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:45.072717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:45.072717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:45.072717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:45.074071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:45.074071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:45.074071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:45.074071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:45.074071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:45.074071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:45.074071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:45.074071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:45.074071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:45.074071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:45.074071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:45.074071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:45.074071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:45.074071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:45.074071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:45.074071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:45.075032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:45.075032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:45.075032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:45.075032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:45.075032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:45.075032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:45.075032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:45.075032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:45.075032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:45.075032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:45.075032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:45.075032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:45.075032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:51:45.075032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:45.077963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 484/951] 10.222.0.76 () {56 vars in 1126 bytes} [Thu May 26 17:51:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:45.084981 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a666c7d-72e5-49e3-a396-8ff6c4ced98b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:45.147924 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 468/952] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:45.164805 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:45.172391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283c7ab-dc60-49b2-bc0b-6b75f2c0f0fb None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:45.258400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a666c7d-72e5-49e3-a396-8ff6c4ced98b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:45.281830 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:45.301712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283c7ab-dc60-49b2-bc0b-6b75f2c0f0fb None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:45.308327 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c9c7d-bc22-4b22-9a4f-17199bfdb054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:45.309081 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c9c7d-bc22-4b22-9a4f-17199bfdb054 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:45.309370 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c9c7d-bc22-4b22-9a4f-17199bfdb054 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:45.309707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c9c7d-bc22-4b22-9a4f-17199bfdb054 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:45.310973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c9c7d-bc22-4b22-9a4f-17199bfdb054 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:45.311304 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c9c7d-bc22-4b22-9a4f-17199bfdb054 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:45.313236 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c9c7d-bc22-4b22-9a4f-17199bfdb054 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:45.327829 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 485/953] 10.222.0.76 () {56 vars in 1122 bytes} [Thu May 26 17:51:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:45.328960 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c9c7d-bc22-4b22-9a4f-17199bfdb054 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:45.343625 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633545a-ddb5-45fd-9e7e-49c0ba97f945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:45.344818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633545a-ddb5-45fd-9e7e-49c0ba97f945 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:45.345667 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633545a-ddb5-45fd-9e7e-49c0ba97f945 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:45.346260 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633545a-ddb5-45fd-9e7e-49c0ba97f945 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:45.359011 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633545a-ddb5-45fd-9e7e-49c0ba97f945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:51:45.368930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-6633545a-ddb5-45fd-9e7e-49c0ba97f945 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:51:45.368930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:45.368930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:45.368930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:45.368930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:45.368930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:45.368930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:45.368930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:45.368930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:45.368930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:45.368930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:45.368930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:45.368930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:51:45.368930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:51:45.368930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:51:45.368930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:51:45.369967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:51:45.369967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:51:45.369967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:51:45.369967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:51:45.369967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:45.369967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:45.369967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:45.369967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:45.369967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:45.369967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:45.369967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:45.369967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:45.369967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:45.369967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:45.369967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:45.369967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:45.370869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:45.370869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:45.370869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:45.370869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:45.370869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:51:45.370869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:51:45.370869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:51:45.370869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:51:45.370869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:51:45.370869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:51:45.370869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:51:45.370869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:45.379756 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 469/954] 10.222.0.76 () {56 vars in 1120 bytes} [Thu May 26 17:51:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:51:45.381495 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633545a-ddb5-45fd-9e7e-49c0ba97f945 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:45.438650 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c9c7d-bc22-4b22-9a4f-17199bfdb054 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:45.454698 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:45.481898 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345fc6e-a009-4072-b5f5-6df9aaefc812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:45.482631 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345fc6e-a009-4072-b5f5-6df9aaefc812 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:45.482915 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345fc6e-a009-4072-b5f5-6df9aaefc812 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:45.483205 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345fc6e-a009-4072-b5f5-6df9aaefc812 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:45.484688 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345fc6e-a009-4072-b5f5-6df9aaefc812 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:45.485201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345fc6e-a009-4072-b5f5-6df9aaefc812 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:45.488499 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345fc6e-a009-4072-b5f5-6df9aaefc812 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:45.497919 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 486/955] 10.222.0.76 () {56 vars in 1122 bytes} [Thu May 26 17:51:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:45.505985 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345fc6e-a009-4072-b5f5-6df9aaefc812 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:45.518470 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633545a-ddb5-45fd-9e7e-49c0ba97f945 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:45.575698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e8c3a-eaeb-4828-b343-b7118b490c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:45.576787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e8c3a-eaeb-4828-b343-b7118b490c89 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:45.577241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e8c3a-eaeb-4828-b343-b7118b490c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:45.577771 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e8c3a-eaeb-4828-b343-b7118b490c89 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:45.578914 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e8c3a-eaeb-4828-b343-b7118b490c89 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:45.579932 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e8c3a-eaeb-4828-b343-b7118b490c89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:45.583173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e8c3a-eaeb-4828-b343-b7118b490c89 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:45.602225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 470/956] 10.222.0.76 () {56 vars in 1115 bytes} [Thu May 26 17:51:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:45.605652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e8c3a-eaeb-4828-b343-b7118b490c89 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:45.620431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:45.686936 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345fc6e-a009-4072-b5f5-6df9aaefc812 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:45.723220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e8c3a-eaeb-4828-b343-b7118b490c89 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:45.742977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619978e6-26c7-4e53-b81e-ca6205f62a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:45.744129 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619978e6-26c7-4e53-b81e-ca6205f62a6c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:45.744606 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619978e6-26c7-4e53-b81e-ca6205f62a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:45.745076 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619978e6-26c7-4e53-b81e-ca6205f62a6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:45.746559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619978e6-26c7-4e53-b81e-ca6205f62a6c None admin] RBAC: Adding query filter params (interface=public, service_id=9fd75d08b72241aa8e65ea949747e690, region_id=RegionOne) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:45.747070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619978e6-26c7-4e53-b81e-ca6205f62a6c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:45.749582 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619978e6-26c7-4e53-b81e-ca6205f62a6c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:45.766618 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 487/957] 10.222.0.76 () {56 vars in 1251 bytes} [Thu May 26 17:51:45 2022] GET /identity/v3/endpoints?service_id=9fd75d08b72241aa8e65ea949747e690&interface=public®ion_id=RegionOne => generated 194 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:45.769306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619978e6-26c7-4e53-b81e-ca6205f62a6c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:45.770818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb3124-13ab-4721-94b9-6eb86a5537f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:45.771890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb3124-13ab-4721-94b9-6eb86a5537f6 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:45.772381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb3124-13ab-4721-94b9-6eb86a5537f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:45.772857 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb3124-13ab-4721-94b9-6eb86a5537f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:45.785383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb3124-13ab-4721-94b9-6eb86a5537f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:45.787585 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb3124-13ab-4721-94b9-6eb86a5537f6 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:45.807741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-88eb3124-13ab-4721-94b9-6eb86a5537f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:45.807741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:45.807741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:45.807741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:45.807741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:45.807741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:45.807741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:45.807741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:45.807741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:45.807741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:45.807741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:45.807741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:45.807741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:45.807741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:45.807741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:45.808698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:45.808698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:45.808698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:45.808698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:45.808698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:45.808698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:45.808698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:45.808698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:45.808698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:45.808698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:45.808698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:45.808698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:45.808698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:45.808698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:45.808698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:45.808698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:45.808698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:45.809605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:45.809605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:45.809605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:45.809605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:45.816805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 471/958] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:45.818555 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb3124-13ab-4721-94b9-6eb86a5537f6 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:45.851302 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:45.885476 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:45.932296 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb3124-13ab-4721-94b9-6eb86a5537f6 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:45.956228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619978e6-26c7-4e53-b81e-ca6205f62a6c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:45.968995 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70878154-3a70-4e7f-9bf4-0043fb28a66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:45.970078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70878154-3a70-4e7f-9bf4-0043fb28a66e None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:45.971224 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70878154-3a70-4e7f-9bf4-0043fb28a66e None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:45.971702 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70878154-3a70-4e7f-9bf4-0043fb28a66e None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:45.972631 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70878154-3a70-4e7f-9bf4-0043fb28a66e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:45.973140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70878154-3a70-4e7f-9bf4-0043fb28a66e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:45.974865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70878154-3a70-4e7f-9bf4-0043fb28a66e None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:45.992759 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 472/959] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:45.994514 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70878154-3a70-4e7f-9bf4-0043fb28a66e None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:46.016632 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc72745-86b1-4b31-a305-e5ad3fb92e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:46.018281 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc72745-86b1-4b31-a305-e5ad3fb92e7c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:46.018756 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc72745-86b1-4b31-a305-e5ad3fb92e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:46.019229 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc72745-86b1-4b31-a305-e5ad3fb92e7c None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:46.020531 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc72745-86b1-4b31-a305-e5ad3fb92e7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:46.022902 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc72745-86b1-4b31-a305-e5ad3fb92e7c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:46.041254 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 488/960] 10.222.0.76 () {56 vars in 1087 bytes} [Thu May 26 17:51:45 2022] GET /identity/v3/services => generated 1497 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 17:51:46.118985 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70878154-3a70-4e7f-9bf4-0043fb28a66e None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:46.143859 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:46.160536 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb6fa2-db56-4681-b43a-224a8e5ea835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:46.161790 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb6fa2-db56-4681-b43a-224a8e5ea835 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:46.161929 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb6fa2-db56-4681-b43a-224a8e5ea835 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:46.162348 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb6fa2-db56-4681-b43a-224a8e5ea835 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:46.170958 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb6fa2-db56-4681-b43a-224a8e5ea835 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:46.172452 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb6fa2-db56-4681-b43a-224a8e5ea835 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:46.189177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-6afb6fa2-db56-4681-b43a-224a8e5ea835 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:51:46.189177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:46.189177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:46.189177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:46.189177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:46.189177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:46.189177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:46.189177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:46.189177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:46.189177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:46.189177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:46.189177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:46.189177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:51:46.189177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:51:46.189177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:51:46.189177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:51:46.191061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:46.191061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:46.191061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:51:46.191061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:46.191061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:51:46.191061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:46.191061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:46.191061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:46.191061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:46.191061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:46.191061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:46.191061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:46.191061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:46.191061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:46.191061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:46.191061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:46.191061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:46.191061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:46.192919 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:46.192919 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:46.192919 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:51:46.192919 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:51:46.192919 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:51:46.192919 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:51:46.192919 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:51:46.192919 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:51:46.192919 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 17:51:46.192919 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:46.198260 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 473/961] 10.222.0.76 () {56 vars in 1117 bytes} [Thu May 26 17:51:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:46.200236 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc72745-86b1-4b31-a305-e5ad3fb92e7c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:46.293509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b48306-dbc1-4a8a-b61c-7b1253506485 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:46.293509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b48306-dbc1-4a8a-b61c-7b1253506485 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:46.293509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b48306-dbc1-4a8a-b61c-7b1253506485 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:46.293509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 474/962] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:46.300014 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c0d90-0df4-4e5d-9983-cfd4dbfcdf8f None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:46.300269 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c0d90-0df4-4e5d-9983-cfd4dbfcdf8f None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:46.300552 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c0d90-0df4-4e5d-9983-cfd4dbfcdf8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:46.320858 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-ec0c0d90-0df4-4e5d-9983-cfd4dbfcdf8f None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:46.322089 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-ec0c0d90-0df4-4e5d-9983-cfd4dbfcdf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:46.365634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc72745-86b1-4b31-a305-e5ad3fb92e7c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:46.370826 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:46.406530 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8878ef7f-586d-415a-95a8-055e86e18e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:46.407206 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878ef7f-586d-415a-95a8-055e86e18e6b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:46.407487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878ef7f-586d-415a-95a8-055e86e18e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:46.408579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878ef7f-586d-415a-95a8-055e86e18e6b None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:46.409576 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8878ef7f-586d-415a-95a8-055e86e18e6b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:46.409951 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8878ef7f-586d-415a-95a8-055e86e18e6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:46.411641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8878ef7f-586d-415a-95a8-055e86e18e6b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:46.418500 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 475/963] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:51:46.438329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 489/964] 10.222.0.76 () {56 vars in 1149 bytes} [Thu May 26 17:51:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:46.441763 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c0d90-0df4-4e5d-9983-cfd4dbfcdf8f None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:46.445382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8878ef7f-586d-415a-95a8-055e86e18e6b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:46.551660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8878ef7f-586d-415a-95a8-055e86e18e6b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:46.560077 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c0d90-0df4-4e5d-9983-cfd4dbfcdf8f None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:46.573292 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:46.587937 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff5515-c44f-430b-ab96-e773bab58f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:46.588615 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff5515-c44f-430b-ab96-e773bab58f38 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:46.588973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff5515-c44f-430b-ab96-e773bab58f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:46.589150 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff5515-c44f-430b-ab96-e773bab58f38 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:46.589862 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff5515-c44f-430b-ab96-e773bab58f38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:46.591320 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff5515-c44f-430b-ab96-e773bab58f38 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:46.604668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b99653-ad00-4a8a-8a30-1e6c65bc49e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:46.605496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b99653-ad00-4a8a-8a30-1e6c65bc49e7 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:46.606033 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b99653-ad00-4a8a-8a30-1e6c65bc49e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:46.606376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b99653-ad00-4a8a-8a30-1e6c65bc49e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:46.607767 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.sql.core [None req-9dff5515-c44f-430b-ab96-e773bab58f38 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 17:51:46.607767 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 26 17:51:46.607767 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [parameters: {'user_id': '71c53dc35b08477b8703215915eb8755', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 26 17:51:46.607767 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80261) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 17:51:46.616335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-9dff5515-c44f-430b-ab96-e773bab58f38 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 26 17:51:46.616335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:46.616335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 17:51:46.616335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 17:51:46.616335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 17:51:46.616335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 17:51:46.616335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 17:51:46.616335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application result = self._query(query) May 26 17:51:46.616335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 17:51:46.616335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application conn.query(q) May 26 17:51:46.616335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 17:51:46.616335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 17:51:46.616335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 17:51:46.616335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application result.read() May 26 17:51:46.616335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 17:51:46.617264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 17:51:46.617264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 17:51:46.617264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 17:51:46.617264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 17:51:46.617264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 17:51:46.617264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 17:51:46.617264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 17:51:46.617264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 17:51:46.617264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:46.617264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 17:51:46.617264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:46.617264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:46.617264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 17:51:46.617264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 17:51:46.617264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 26 17:51:46.617264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 26 17:51:46.617264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 17:51:46.617264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application next(self.gen) May 26 17:51:46.618251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 17:51:46.618251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application yield resource May 26 17:51:46.618251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 17:51:46.618251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application next(self.gen) May 26 17:51:46.618251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 17:51:46.618251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.session.rollback() May 26 17:51:46.618251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 17:51:46.618251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.force_reraise() May 26 17:51:46.618251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 17:51:46.618251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise self.value May 26 17:51:46.618251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 17:51:46.618251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 17:51:46.618251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 17:51:46.618251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application session.commit() May 26 17:51:46.618251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 17:51:46.618251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 17:51:46.618251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 17:51:46.619269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._prepare_impl() May 26 17:51:46.619269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 17:51:46.619269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.session.flush() May 26 17:51:46.619269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 17:51:46.619269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._flush(objects) May 26 17:51:46.619269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 17:51:46.619269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 17:51:46.619269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 17:51:46.619269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application compat.raise_( May 26 17:51:46.619269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 17:51:46.619269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception May 26 17:51:46.619269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 17:51:46.619269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application flush_context.execute() May 26 17:51:46.619269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 17:51:46.619269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rec.execute(self) May 26 17:51:46.619269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 17:51:46.619269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 17:51:46.620149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 17:51:46.620149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 17:51:46.620149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 26 17:51:46.620149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application result = connection._execute_20( May 26 17:51:46.620149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 17:51:46.620149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 17:51:46.620149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 17:51:46.620149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 17:51:46.620149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 17:51:46.620149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 17:51:46.620149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 17:51:46.620149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 17:51:46.620149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 17:51:46.620149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 17:51:46.620149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 17:51:46.620149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception May 26 17:51:46.620997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 17:51:46.620997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 17:51:46.620997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 17:51:46.620997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 17:51:46.620997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 17:51:46.620997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application result = self._query(query) May 26 17:51:46.620997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 17:51:46.620997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application conn.query(q) May 26 17:51:46.620997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 17:51:46.620997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 17:51:46.620997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 17:51:46.620997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application result.read() May 26 17:51:46.620997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 17:51:46.620997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 17:51:46.620997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 17:51:46.620997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 17:51:46.621843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 17:51:46.621843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 17:51:46.621843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 17:51:46.621843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 17:51:46.621843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 17:51:46.621843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 26 17:51:46.621843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [parameters: {'user_id': '71c53dc35b08477b8703215915eb8755', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 26 17:51:46.621843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 17:51:46.621843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:46.621843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:46.621843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:46.621843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:46.621843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:46.621843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:46.621843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:46.622807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:46.622807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:46.622807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:46.622807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:46.622807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:46.622807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:46.622807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:46.622807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 26 17:51:46.622807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 26 17:51:46.622807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:46.622807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:46.622807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:51:46.622807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:46.622807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:51:46.622807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:46.622807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 26 17:51:46.622807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 26 17:51:46.623757 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 26 17:51:46.623757 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 26 17:51:46.623757 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 17:51:46.623757 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 17:51:46.623757 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 26 17:51:46.623757 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:46.625973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 490/965] 10.222.0.76 () {60 vars in 1131 bytes} [Thu May 26 17:51:46 2022] POST /identity/v3/users => generated 160 bytes in 181 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:46.632725 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff5515-c44f-430b-ab96-e773bab58f38 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:46.671170 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:46.712911 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b99653-ad00-4a8a-8a30-1e6c65bc49e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:46.715885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b99653-ad00-4a8a-8a30-1e6c65bc49e7 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:46.793288 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff5515-c44f-430b-ab96-e773bab58f38 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:46.841973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9d6d7-3b8e-482a-9a44-9a78f4833984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:46.842901 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9d6d7-3b8e-482a-9a44-9a78f4833984 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:46.843290 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9d6d7-3b8e-482a-9a44-9a78f4833984 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:46.843675 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9d6d7-3b8e-482a-9a44-9a78f4833984 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:46.852535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9d6d7-3b8e-482a-9a44-9a78f4833984 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:46.854369 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9d6d7-3b8e-482a-9a44-9a78f4833984 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:46.870433 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 476/966] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:51:46 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:51:46.870821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-25a9d6d7-3b8e-482a-9a44-9a78f4833984 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 17:51:46.870821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:46.870821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:46.870821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:46.870821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:46.870821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:46.870821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:46.870821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:46.870821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:46.870821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:46.870821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:46.870821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:46.870821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:51:46.870821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:51:46.870821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:51:46.870821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:51:46.871927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:46.871927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:46.871927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:51:46.871927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:46.871927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:51:46.871927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:46.871927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:46.871927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:46.871927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:46.871927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:46.871927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:46.871927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:46.871927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:46.871927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:46.871927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:46.871927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:46.871927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:46.871927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:46.873093 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:46.873093 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:46.873093 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:51:46.873093 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:51:46.873093 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:51:46.873093 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:51:46.873093 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:51:46.873093 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:51:46.873093 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 17:51:46.873093 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:46.874706 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b99653-ad00-4a8a-8a30-1e6c65bc49e7 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:46.879741 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 491/967] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:51:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:51:46.880791 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9d6d7-3b8e-482a-9a44-9a78f4833984 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:46.889248 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:46.987485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:47.012266 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9d6d7-3b8e-482a-9a44-9a78f4833984 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:47.012754 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b99653-ad00-4a8a-8a30-1e6c65bc49e7 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:47.055411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66411f9b-60cb-4c08-8b2e-c82312a28486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:47.056547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66411f9b-60cb-4c08-8b2e-c82312a28486 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:47.056982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66411f9b-60cb-4c08-8b2e-c82312a28486 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:47.057368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66411f9b-60cb-4c08-8b2e-c82312a28486 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:47.061424 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c375c-abc3-4b64-9965-3a15b5f05ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:47.062328 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c375c-abc3-4b64-9965-3a15b5f05ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:47.062775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c375c-abc3-4b64-9965-3a15b5f05ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:47.063207 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c375c-abc3-4b64-9965-3a15b5f05ef0 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:47.064673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c375c-abc3-4b64-9965-3a15b5f05ef0 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:47.065128 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c375c-abc3-4b64-9965-3a15b5f05ef0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:47.067358 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c375c-abc3-4b64-9965-3a15b5f05ef0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:47.070800 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66411f9b-60cb-4c08-8b2e-c82312a28486 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:47.072460 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66411f9b-60cb-4c08-8b2e-c82312a28486 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:47.092132 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-66411f9b-60cb-4c08-8b2e-c82312a28486 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:47.092132 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:47.092132 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:47.092132 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:47.092132 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:47.092132 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:47.092132 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:47.092132 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:47.092132 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:47.092132 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:47.092132 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:47.092132 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:47.092132 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:47.092132 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:47.092132 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:47.093271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:47.093271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:47.093271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:47.093271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:47.093271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:47.093271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:47.093271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:47.093271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:47.093271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:47.093271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:47.093271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:47.093271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:47.093271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:47.093271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:47.093271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:47.093271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:47.093271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:47.094441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:47.094441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:47.094441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:47.094441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:47.100297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 477/968] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:47.104161 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66411f9b-60cb-4c08-8b2e-c82312a28486 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:47.122281 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 492/969] 10.222.0.76 () {56 vars in 1101 bytes} [Thu May 26 17:51:46 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:47.178270 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:47.204310 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05101564-6ee5-4779-b5f1-f69a2fe556b8 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:47.205453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05101564-6ee5-4779-b5f1-f69a2fe556b8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:47.205453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05101564-6ee5-4779-b5f1-f69a2fe556b8 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:47.207532 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 493/970] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:47.213676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b2702-25a5-40b2-9426-e5626b9a7f58 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:47.213676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b2702-25a5-40b2-9426-e5626b9a7f58 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:47.214021 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b2702-25a5-40b2-9426-e5626b9a7f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:47.229987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66411f9b-60cb-4c08-8b2e-c82312a28486 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:47.232000 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3a5b2702-25a5-40b2-9426-e5626b9a7f58 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:47.234180 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3a5b2702-25a5-40b2-9426-e5626b9a7f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:47.282853 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2121d1-d094-4fad-b00f-6ebeaaae8d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:47.283567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2121d1-d094-4fad-b00f-6ebeaaae8d0e None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:47.283860 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2121d1-d094-4fad-b00f-6ebeaaae8d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:47.284135 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2121d1-d094-4fad-b00f-6ebeaaae8d0e None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:47.284925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2121d1-d094-4fad-b00f-6ebeaaae8d0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:47.285183 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2121d1-d094-4fad-b00f-6ebeaaae8d0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:47.287117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2121d1-d094-4fad-b00f-6ebeaaae8d0e None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:47.305355 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 478/971] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:47.308200 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2121d1-d094-4fad-b00f-6ebeaaae8d0e None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:47.326270 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 494/972] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:51:47.344951 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b2702-25a5-40b2-9426-e5626b9a7f58 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:47.441353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:47.448293 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2121d1-d094-4fad-b00f-6ebeaaae8d0e None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:47.476220 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b2702-25a5-40b2-9426-e5626b9a7f58 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:47.481254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:47.494605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776f1d0-7bf4-4819-a740-702240de3783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:47.495415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776f1d0-7bf4-4819-a740-702240de3783 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:47.495708 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776f1d0-7bf4-4819-a740-702240de3783 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:47.495984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776f1d0-7bf4-4819-a740-702240de3783 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:47.504543 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776f1d0-7bf4-4819-a740-702240de3783 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:47.506018 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776f1d0-7bf4-4819-a740-702240de3783 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:47.517376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532eb47-3d43-42eb-852b-54575fa77072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:47.518250 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532eb47-3d43-42eb-852b-54575fa77072 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:47.518544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532eb47-3d43-42eb-852b-54575fa77072 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:47.518825 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532eb47-3d43-42eb-852b-54575fa77072 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:47.519561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b532eb47-3d43-42eb-852b-54575fa77072 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:47.520953 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b532eb47-3d43-42eb-852b-54575fa77072 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:47.523147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-3776f1d0-7bf4-4819-a740-702240de3783 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:47.523147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:47.523147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:47.523147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:47.523147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:47.523147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:47.523147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:47.523147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:47.523147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:47.523147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:47.523147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:47.523147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:47.523147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:51:47.523147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:51:47.523147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:51:47.524206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:51:47.524206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:47.524206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:47.524206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:47.524206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:47.524206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:47.524206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:47.524206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:47.524206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:47.524206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:47.524206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:47.524206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:47.524206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:47.524206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:47.524206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:47.524206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:47.524206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:47.525383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:51:47.525383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:51:47.525383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:51:47.525383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:51:47.525383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:51:47.525383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:51:47.525383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:47.525383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:47.527603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 479/973] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:47.533493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776f1d0-7bf4-4819-a740-702240de3783 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:47.538033 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 495/974] 10.222.0.76 () {60 vars in 1141 bytes} [Thu May 26 17:51:47 2022] POST /identity/v3/services => generated 242 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:51:47.656883 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776f1d0-7bf4-4819-a740-702240de3783 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:47.703019 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4882dc-b699-45d3-bf3b-b7827662dbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:47.703708 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4882dc-b699-45d3-bf3b-b7827662dbeb None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:47.703996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4882dc-b699-45d3-bf3b-b7827662dbeb None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:47.704276 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4882dc-b699-45d3-bf3b-b7827662dbeb None admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:47.705204 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4882dc-b699-45d3-bf3b-b7827662dbeb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:47.705525 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4882dc-b699-45d3-bf3b-b7827662dbeb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:47.707059 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4882dc-b699-45d3-bf3b-b7827662dbeb None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:47.721197 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 480/975] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:51:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:47.727695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532eb47-3d43-42eb-852b-54575fa77072 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:47.857559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532eb47-3d43-42eb-852b-54575fa77072 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:47.892858 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:47.899917 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb453c30-e81e-4a3d-b01b-4c52f33ef93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:47.901044 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb453c30-e81e-4a3d-b01b-4c52f33ef93c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:47.901663 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb453c30-e81e-4a3d-b01b-4c52f33ef93c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:47.902034 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb453c30-e81e-4a3d-b01b-4c52f33ef93c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:47.902991 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb453c30-e81e-4a3d-b01b-4c52f33ef93c None admin] RBAC: Adding query filter params (role.id=04c312d9dc3249fe8fbff78befef65f9, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=3e06d1b0e1a9472db764f4be7e42fdd9) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:47.903403 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb453c30-e81e-4a3d-b01b-4c52f33ef93c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:47.904955 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb453c30-e81e-4a3d-b01b-4c52f33ef93c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:47.918307 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 496/976] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:51:47 2022] GET /identity/v3/role_assignments?user.id=3e06d1b0e1a9472db764f4be7e42fdd9&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=04c312d9dc3249fe8fbff78befef65f9 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:48.440270 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e90e4d-1cbd-4dad-8f6d-624a6d12eb8f None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:48.440908 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e90e4d-1cbd-4dad-8f6d-624a6d12eb8f None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:48.440908 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e90e4d-1cbd-4dad-8f6d-624a6d12eb8f None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:48.442229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 481/977] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:48.448087 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056767d4-1bb6-493e-b912-3fabd485b594 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:48.448523 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056767d4-1bb6-493e-b912-3fabd485b594 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:48.448701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056767d4-1bb6-493e-b912-3fabd485b594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:48.466693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-056767d4-1bb6-493e-b912-3fabd485b594 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:48.467891 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-056767d4-1bb6-493e-b912-3fabd485b594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:48.537610 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 497/978] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:48.559673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e90e4d-1cbd-4dad-8f6d-624a6d12eb8f None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:48.603071 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:48.667743 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e90e4d-1cbd-4dad-8f6d-624a6d12eb8f None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:48.702653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26bef74-ac43-4771-81ca-de6ae65e380a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:48.703399 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bef74-ac43-4771-81ca-de6ae65e380a None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:48.703687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bef74-ac43-4771-81ca-de6ae65e380a None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:48.703964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bef74-ac43-4771-81ca-de6ae65e380a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:48.704744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26bef74-ac43-4771-81ca-de6ae65e380a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:48.706225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26bef74-ac43-4771-81ca-de6ae65e380a None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:48.719280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-e26bef74-ac43-4771-81ca-de6ae65e380a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:51:48.719280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:48.719280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:48.719280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:48.719280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:48.719280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:48.719280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:48.719280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:48.719280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:51:48.719280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:48.719280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:48.719280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:48.719280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:48.719280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:48.719280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:48.719280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:48.719280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:48.720982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:48.720982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:48.720982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:48.720982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:48.720982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:48.720982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:48.720982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:48.720982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:48.720982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:48.720982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:48.720982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:48.720982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:48.720982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:48.720982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:48.720982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:48.720982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:48.722673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:48.722673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:48.722673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:48.722673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:48.722673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:48.722673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:48.722673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:48.722673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:48.722673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:48.722673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:48.722673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:48.722673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:48.722673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:51:48.722673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:48.726187 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 482/979] 10.222.0.76 () {56 vars in 1126 bytes} [Thu May 26 17:51:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:48.727382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056767d4-1bb6-493e-b912-3fabd485b594 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:48.837951 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056767d4-1bb6-493e-b912-3fabd485b594 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:48.870333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:48.888761 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda82ae6-6b0f-4232-ba7c-f2fc51015cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:48.889454 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda82ae6-6b0f-4232-ba7c-f2fc51015cea None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:48.889736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda82ae6-6b0f-4232-ba7c-f2fc51015cea None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:48.890069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda82ae6-6b0f-4232-ba7c-f2fc51015cea None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:48.890764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda82ae6-6b0f-4232-ba7c-f2fc51015cea None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:48.891071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda82ae6-6b0f-4232-ba7c-f2fc51015cea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:48.893343 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda82ae6-6b0f-4232-ba7c-f2fc51015cea None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:48.907688 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 498/980] 10.222.0.76 () {56 vars in 1122 bytes} [Thu May 26 17:51:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:48.909522 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26bef74-ac43-4771-81ca-de6ae65e380a None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:49.012326 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26bef74-ac43-4771-81ca-de6ae65e380a None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:49.013878 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:49.049476 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e634499-2916-4705-af11-8637242dac23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:49.050143 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e634499-2916-4705-af11-8637242dac23 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:49.050459 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e634499-2916-4705-af11-8637242dac23 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:49.050738 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e634499-2916-4705-af11-8637242dac23 None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:49.051445 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e634499-2916-4705-af11-8637242dac23 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:49.051755 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e634499-2916-4705-af11-8637242dac23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:49.053112 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e634499-2916-4705-af11-8637242dac23 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:49.062822 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 483/981] 10.222.0.76 () {56 vars in 1122 bytes} [Thu May 26 17:51:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:49.063951 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda82ae6-6b0f-4232-ba7c-f2fc51015cea None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:49.180415 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda82ae6-6b0f-4232-ba7c-f2fc51015cea None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:49.214573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69ee1e-48a3-49e0-9790-59e2b88f9787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:49.215223 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69ee1e-48a3-49e0-9790-59e2b88f9787 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:49.215508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69ee1e-48a3-49e0-9790-59e2b88f9787 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:49.215783 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69ee1e-48a3-49e0-9790-59e2b88f9787 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:49.216524 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69ee1e-48a3-49e0-9790-59e2b88f9787 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:49.217893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69ee1e-48a3-49e0-9790-59e2b88f9787 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:49.243931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 499/982] 10.222.0.76 () {60 vars in 1144 bytes} [Thu May 26 17:51:49 2022] POST /identity/v3/endpoints => generated 349 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:51:49.852528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcd7d9-58c2-4034-b274-f6cb2c65b0e7 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:49.853074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcd7d9-58c2-4034-b274-f6cb2c65b0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:49.853074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcd7d9-58c2-4034-b274-f6cb2c65b0e7 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:49.854738 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 484/983] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:49.859950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949987b0-a37c-464c-9770-e53931d784b3 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:49.860393 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949987b0-a37c-464c-9770-e53931d784b3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:49.860585 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949987b0-a37c-464c-9770-e53931d784b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:49.879911 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-949987b0-a37c-464c-9770-e53931d784b3 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:49.881136 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-949987b0-a37c-464c-9770-e53931d784b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:49.927076 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:49.965057 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 500/984] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:49.988714 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcd7d9-58c2-4034-b274-f6cb2c65b0e7 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:50.105067 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcd7d9-58c2-4034-b274-f6cb2c65b0e7 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:50.151536 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16ec84-f9cc-4026-b9a7-c4853dcedf87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:50.154131 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16ec84-f9cc-4026-b9a7-c4853dcedf87 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:50.154131 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16ec84-f9cc-4026-b9a7-c4853dcedf87 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:50.154131 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16ec84-f9cc-4026-b9a7-c4853dcedf87 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:50.154131 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc16ec84-f9cc-4026-b9a7-c4853dcedf87 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:50.154958 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc16ec84-f9cc-4026-b9a7-c4853dcedf87 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:50.168050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-cc16ec84-f9cc-4026-b9a7-c4853dcedf87 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 17:51:50.168050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:50.168050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:50.168050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:50.168050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:50.168050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:50.168050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:50.168050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:50.168050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 17:51:50.168050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:50.168050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:50.168050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:50.168050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:50.168050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:50.168050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:50.168050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:50.168050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:50.169173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:50.169173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:50.169173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:50.169173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:50.169173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:50.169173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:50.169173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:50.169173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:50.169173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:50.169173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:50.169173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:50.169173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:50.169173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:50.169173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:50.169173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:50.169173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:50.170054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:50.170054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:50.170054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:50.170054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:50.170054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:50.170054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:50.170054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:50.170054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:50.170054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:50.170054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:50.170054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:50.170054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:50.170054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 17:51:50.170054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:50.171704 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 485/985] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:50.176041 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949987b0-a37c-464c-9770-e53931d784b3 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:50.257873 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57b6b0-8127-4ab8-8478-ac52effecc63 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:50.258448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57b6b0-8127-4ab8-8478-ac52effecc63 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:50.258808 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57b6b0-8127-4ab8-8478-ac52effecc63 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:50.260842 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 486/986] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:50.266426 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d3504-f2aa-4215-9b64-126e31659220 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:50.266786 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d3504-f2aa-4215-9b64-126e31659220 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:50.267069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d3504-f2aa-4215-9b64-126e31659220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:50.284829 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-a59d3504-f2aa-4215-9b64-126e31659220 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:50.286679 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-a59d3504-f2aa-4215-9b64-126e31659220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:50.301155 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949987b0-a37c-464c-9770-e53931d784b3 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:50.313875 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:50.345110 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb097b-5a82-4b87-b199-24d44b788488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:50.346769 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb097b-5a82-4b87-b199-24d44b788488 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:50.347191 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb097b-5a82-4b87-b199-24d44b788488 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:50.347668 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb097b-5a82-4b87-b199-24d44b788488 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:50.348660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb097b-5a82-4b87-b199-24d44b788488 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:50.349097 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb097b-5a82-4b87-b199-24d44b788488 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:50.350759 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb097b-5a82-4b87-b199-24d44b788488 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:50.362724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 501/987] 10.222.0.76 () {56 vars in 1112 bytes} [Thu May 26 17:51:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:50.363660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb097b-5a82-4b87-b199-24d44b788488 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:50.370835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 487/988] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:50 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:51:50.394016 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d3504-f2aa-4215-9b64-126e31659220 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:50.409361 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:50.496877 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb097b-5a82-4b87-b199-24d44b788488 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:50.510939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d3504-f2aa-4215-9b64-126e31659220 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:50.537211 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a901af-5c8d-4f10-9ef3-73917976394c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:50.538053 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a901af-5c8d-4f10-9ef3-73917976394c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:50.538479 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a901af-5c8d-4f10-9ef3-73917976394c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:50.538766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a901af-5c8d-4f10-9ef3-73917976394c None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:50.539477 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a901af-5c8d-4f10-9ef3-73917976394c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:50.539788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a901af-5c8d-4f10-9ef3-73917976394c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:50.541173 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a901af-5c8d-4f10-9ef3-73917976394c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:50.547048 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 502/989] 10.222.0.76 () {56 vars in 1112 bytes} [Thu May 26 17:51:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:50.552770 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a901af-5c8d-4f10-9ef3-73917976394c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:50.556751 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbff1b-7bf7-4b11-9fde-da9eabf8df72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:50.557563 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbff1b-7bf7-4b11-9fde-da9eabf8df72 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:50.557845 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbff1b-7bf7-4b11-9fde-da9eabf8df72 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:50.558110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbff1b-7bf7-4b11-9fde-da9eabf8df72 None admin] PATH_INFO: `/v3/services/image` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:50.558856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbff1b-7bf7-4b11-9fde-da9eabf8df72 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:50.560252 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbff1b-7bf7-4b11-9fde-da9eabf8df72 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:50.580923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-0cfbff1b-7bf7-4b11-9fde-da9eabf8df72 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 17:51:50.580923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:50.580923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:50.580923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:50.580923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:50.580923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:50.580923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:50.580923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:50.580923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 17:51:50.580923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:50.580923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:50.580923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:50.580923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:50.580923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:50.580923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:50.580923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:50.580923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:50.582211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:50.582211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:50.582211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:50.582211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:50.582211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:50.582211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:50.582211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:50.582211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:50.582211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:50.582211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:50.582211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:50.582211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:50.582211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:50.582211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:50.582211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:50.582211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:50.583069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:50.583069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:50.583069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:50.583069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:50.583069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:50.583069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:50.583069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:50.583069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:50.583069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:50.583069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:50.583069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:50.583069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:50.583069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 17:51:50.583069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:50.587629 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 488/990] 10.222.0.76 () {56 vars in 1105 bytes} [Thu May 26 17:51:50 2022] GET /identity/v3/services/image => generated 86 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:51:50.588376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbff1b-7bf7-4b11-9fde-da9eabf8df72 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:50.617745 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:50.679997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:50.683178 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a901af-5c8d-4f10-9ef3-73917976394c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:50.712927 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbff1b-7bf7-4b11-9fde-da9eabf8df72 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:50.726658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b31ac8-f1ca-4072-80f1-d735149e1cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:50.727320 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b31ac8-f1ca-4072-80f1-d735149e1cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:50.727603 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b31ac8-f1ca-4072-80f1-d735149e1cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:50.727879 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b31ac8-f1ca-4072-80f1-d735149e1cc4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:50.728663 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b31ac8-f1ca-4072-80f1-d735149e1cc4 None admin] RBAC: Adding query filter params (interface=public, service_id=e734c08bccbb45b3b676b790b1f1eb0c, region_id=RegionOne) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:50.728977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b31ac8-f1ca-4072-80f1-d735149e1cc4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:50.730403 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b31ac8-f1ca-4072-80f1-d735149e1cc4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:50.738805 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 503/991] 10.222.0.76 () {56 vars in 1251 bytes} [Thu May 26 17:51:50 2022] GET /identity/v3/endpoints?service_id=e734c08bccbb45b3b676b790b1f1eb0c&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:50.745297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b31ac8-f1ca-4072-80f1-d735149e1cc4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:50.763414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4c2a9-5e1d-4c24-a6a1-772a27bf401a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:50.764098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4c2a9-5e1d-4c24-a6a1-772a27bf401a None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:50.764386 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4c2a9-5e1d-4c24-a6a1-772a27bf401a None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:50.764662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4c2a9-5e1d-4c24-a6a1-772a27bf401a None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:50.765373 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4c2a9-5e1d-4c24-a6a1-772a27bf401a None admin] RBAC: Adding query filter params (name=image) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:50.765682 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4c2a9-5e1d-4c24-a6a1-772a27bf401a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:50.767729 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4c2a9-5e1d-4c24-a6a1-772a27bf401a None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:50.778257 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 489/992] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:50.780051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4c2a9-5e1d-4c24-a6a1-772a27bf401a None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:50.875161 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b31ac8-f1ca-4072-80f1-d735149e1cc4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:50.894169 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:50.910058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4c2a9-5e1d-4c24-a6a1-772a27bf401a None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:50.922222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909208e-2d15-46a8-b1dc-0ff0aef5dfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:50.922949 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909208e-2d15-46a8-b1dc-0ff0aef5dfac None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:50.923226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909208e-2d15-46a8-b1dc-0ff0aef5dfac None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:50.923561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909208e-2d15-46a8-b1dc-0ff0aef5dfac None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:50.924541 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909208e-2d15-46a8-b1dc-0ff0aef5dfac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:50.925686 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909208e-2d15-46a8-b1dc-0ff0aef5dfac None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:50.942222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 504/993] 10.222.0.76 () {56 vars in 1087 bytes} [Thu May 26 17:51:50 2022] GET /identity/v3/services => generated 1727 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 17:51:50.966214 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a68319-5ead-4be2-b381-a46d1a8f6959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:50.967069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a68319-5ead-4be2-b381-a46d1a8f6959 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:50.967423 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a68319-5ead-4be2-b381-a46d1a8f6959 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:50.967765 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a68319-5ead-4be2-b381-a46d1a8f6959 None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:50.968566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a68319-5ead-4be2-b381-a46d1a8f6959 None admin] RBAC: Adding query filter params (type=image) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:50.968941 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a68319-5ead-4be2-b381-a46d1a8f6959 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:50.971197 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a68319-5ead-4be2-b381-a46d1a8f6959 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:50.978242 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 490/994] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:50 2022] GET /identity/v3/services?type=image => generated 122 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:51.499002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38410b-47ab-4c84-999a-a93587bc10b2 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:51.499291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38410b-47ab-4c84-999a-a93587bc10b2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:51.499572 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38410b-47ab-4c84-999a-a93587bc10b2 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:51.500947 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 505/995] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:51.506255 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a8cc4-cde6-4b95-9b75-e49e73d87bf3 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:51.506548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a8cc4-cde6-4b95-9b75-e49e73d87bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:51.506827 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a8cc4-cde6-4b95-9b75-e49e73d87bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:51.523810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-5d9a8cc4-cde6-4b95-9b75-e49e73d87bf3 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:51.524992 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-5d9a8cc4-cde6-4b95-9b75-e49e73d87bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:51.606519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 491/996] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:51 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:51.626370 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38410b-47ab-4c84-999a-a93587bc10b2 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:51.658439 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:51.748087 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38410b-47ab-4c84-999a-a93587bc10b2 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:51.786005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c990c-702e-419f-ae2a-3be3cb010b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:51.786822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c990c-702e-419f-ae2a-3be3cb010b96 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:51.787102 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c990c-702e-419f-ae2a-3be3cb010b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:51.787374 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c990c-702e-419f-ae2a-3be3cb010b96 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:51.788067 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198c990c-702e-419f-ae2a-3be3cb010b96 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:51.789435 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198c990c-702e-419f-ae2a-3be3cb010b96 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:51.803174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-198c990c-702e-419f-ae2a-3be3cb010b96 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:51:51.803174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:51.803174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:51.803174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:51.803174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:51.803174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:51.803174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:51.803174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:51.803174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:51:51.803174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:51.803174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:51.803174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:51.803174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:51.803174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:51.803174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:51.803174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:51.803174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:51.804451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:51.804451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:51.804451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:51.804451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:51.804451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:51.804451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:51.804451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:51.804451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:51.804451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:51.804451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:51.804451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:51.804451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:51.804451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:51.804451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:51.804451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:51.804451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:51.805317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:51.805317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:51.805317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:51.805317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:51.805317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:51.805317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:51.805317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:51.805317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:51.805317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:51.805317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:51.805317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:51.805317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:51.805317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:51:51.805317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:51.805317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 506/997] 10.222.0.76 () {56 vars in 1126 bytes} [Thu May 26 17:51:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:51.810444 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a8cc4-cde6-4b95-9b75-e49e73d87bf3 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:51.897161 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:51.918561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a8cc4-cde6-4b95-9b75-e49e73d87bf3 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:51.953261 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4898ad-cca0-4d16-aa73-e76333be67a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:51.953921 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4898ad-cca0-4d16-aa73-e76333be67a2 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:51.954214 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4898ad-cca0-4d16-aa73-e76333be67a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:51.955129 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4898ad-cca0-4d16-aa73-e76333be67a2 None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:51.956381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4898ad-cca0-4d16-aa73-e76333be67a2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:51.956690 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4898ad-cca0-4d16-aa73-e76333be67a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:51.958061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4898ad-cca0-4d16-aa73-e76333be67a2 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:51.967724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 492/998] 10.222.0.76 () {56 vars in 1122 bytes} [Thu May 26 17:51:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:51.969061 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c990c-702e-419f-ae2a-3be3cb010b96 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:52.079105 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:52.091513 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c990c-702e-419f-ae2a-3be3cb010b96 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:52.137090 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21a10c-0cf6-4b1d-a458-4a356d77e94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:52.138375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21a10c-0cf6-4b1d-a458-4a356d77e94e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:52.141458 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21a10c-0cf6-4b1d-a458-4a356d77e94e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:52.141458 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21a10c-0cf6-4b1d-a458-4a356d77e94e None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:52.141458 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21a10c-0cf6-4b1d-a458-4a356d77e94e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:52.141458 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21a10c-0cf6-4b1d-a458-4a356d77e94e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:52.144031 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21a10c-0cf6-4b1d-a458-4a356d77e94e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:52.156630 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 507/999] 10.222.0.76 () {56 vars in 1122 bytes} [Thu May 26 17:51:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:52.160363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4898ad-cca0-4d16-aa73-e76333be67a2 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:52.294931 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4898ad-cca0-4d16-aa73-e76333be67a2 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:52.333798 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a76099-863e-473e-a991-dca1fe66936b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:52.334403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a76099-863e-473e-a991-dca1fe66936b None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:52.334692 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a76099-863e-473e-a991-dca1fe66936b None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:52.334961 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a76099-863e-473e-a991-dca1fe66936b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:52.335759 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a76099-863e-473e-a991-dca1fe66936b None admin] RBAC: Adding query filter params (interface=admin, service_id=9fd75d08b72241aa8e65ea949747e690, region_id=RegionOne) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:52.336070 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a76099-863e-473e-a991-dca1fe66936b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:52.337438 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a76099-863e-473e-a991-dca1fe66936b None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:52.347317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 493/1000] 10.222.0.76 () {56 vars in 1249 bytes} [Thu May 26 17:51:52 2022] GET /identity/v3/endpoints?service_id=9fd75d08b72241aa8e65ea949747e690&interface=admin®ion_id=RegionOne => generated 193 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:52.348354 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21a10c-0cf6-4b1d-a458-4a356d77e94e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:52.452179 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21a10c-0cf6-4b1d-a458-4a356d77e94e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:52.486092 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf0a16-04bc-4563-99de-4d4911776b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:52.487036 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf0a16-04bc-4563-99de-4d4911776b68 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:52.487324 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf0a16-04bc-4563-99de-4d4911776b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:52.487602 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf0a16-04bc-4563-99de-4d4911776b68 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:52.488272 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf0a16-04bc-4563-99de-4d4911776b68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:52.489637 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf0a16-04bc-4563-99de-4d4911776b68 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:52.502417 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 508/1001] 10.222.0.76 () {56 vars in 1087 bytes} [Thu May 26 17:51:52 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 17:51:53.269864 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b0f1a-e49b-4724-acb2-05af3b8bf7ae None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:53.270464 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b0f1a-e49b-4724-acb2-05af3b8bf7ae None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:53.270464 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b0f1a-e49b-4724-acb2-05af3b8bf7ae None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:53.271789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 494/1002] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:53.277356 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f819e8-dabb-4e55-9d47-8aa4d02c8f18 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:53.277764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f819e8-dabb-4e55-9d47-8aa4d02c8f18 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:53.277930 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f819e8-dabb-4e55-9d47-8aa4d02c8f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:53.292732 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72f800-1312-4268-a13d-7b8a0c43ad8d None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:53.293059 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72f800-1312-4268-a13d-7b8a0c43ad8d None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:53.293289 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72f800-1312-4268-a13d-7b8a0c43ad8d None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:53.294712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 495/1003] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:51:53.295358 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e0f819e8-dabb-4e55-9d47-8aa4d02c8f18 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:53.296629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e0f819e8-dabb-4e55-9d47-8aa4d02c8f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:53.299867 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63400ac1-c40c-4a56-a741-4f1b64b2e0ef None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:53.300164 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63400ac1-c40c-4a56-a741-4f1b64b2e0ef None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:53.300437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63400ac1-c40c-4a56-a741-4f1b64b2e0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:53.313465 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:53.318866 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-63400ac1-c40c-4a56-a741-4f1b64b2e0ef None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:53.320448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-63400ac1-c40c-4a56-a741-4f1b64b2e0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:53.324966 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:53.378137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 509/1004] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:53.400761 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f819e8-dabb-4e55-9d47-8aa4d02c8f18 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:53.404827 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 496/1005] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:51:53.428639 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63400ac1-c40c-4a56-a741-4f1b64b2e0ef None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:53.512699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f819e8-dabb-4e55-9d47-8aa4d02c8f18 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:53.536850 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63400ac1-c40c-4a56-a741-4f1b64b2e0ef None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:53.550375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abc503-4d99-40d4-a1a9-a624e97a20c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:53.551592 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abc503-4d99-40d4-a1a9-a624e97a20c4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:53.552110 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abc503-4d99-40d4-a1a9-a624e97a20c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:53.552581 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abc503-4d99-40d4-a1a9-a624e97a20c4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:53.553430 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81abc503-4d99-40d4-a1a9-a624e97a20c4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:53.555263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81abc503-4d99-40d4-a1a9-a624e97a20c4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:53.568660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-81abc503-4d99-40d4-a1a9-a624e97a20c4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 17:51:53.568660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:53.568660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:53.568660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:53.568660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:53.568660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:53.568660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:53.568660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:53.568660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 17:51:53.568660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:53.568660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:53.568660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:53.568660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:53.568660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:53.568660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:53.568660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:53.568660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:53.569712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:53.569712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:53.569712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:53.569712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:53.569712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:53.569712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:53.569712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:53.569712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:53.569712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:53.569712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:53.569712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:53.569712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:53.569712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:53.569712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:53.569712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:53.569712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:53.570612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:53.570612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:53.570612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:53.570612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:53.570612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:53.570612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:53.570612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:53.570612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:53.570612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:53.570612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:53.570612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:53.570612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:53.570612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 17:51:53.570612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:53.572645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d405f3-b9aa-4f95-9bce-3a1cd2ded7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:53.572923 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 510/1006] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:51:53.573316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d405f3-b9aa-4f95-9bce-3a1cd2ded7ec None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:53.573602 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d405f3-b9aa-4f95-9bce-3a1cd2ded7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:53.573884 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d405f3-b9aa-4f95-9bce-3a1cd2ded7ec None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:53.574652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d405f3-b9aa-4f95-9bce-3a1cd2ded7ec None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:53.576023 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d405f3-b9aa-4f95-9bce-3a1cd2ded7ec None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:53.579206 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abc503-4d99-40d4-a1a9-a624e97a20c4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:53.586376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:53.593451 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 497/1007] 10.222.0.76 () {60 vars in 1141 bytes} [Thu May 26 17:51:53 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:51:53.607514 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:53.710729 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abc503-4d99-40d4-a1a9-a624e97a20c4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:53.765452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f2236-5f45-462a-a582-2371b5d6d1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:53.766665 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f2236-5f45-462a-a582-2371b5d6d1b0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:53.767198 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f2236-5f45-462a-a582-2371b5d6d1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:53.767682 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f2236-5f45-462a-a582-2371b5d6d1b0 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:53.769323 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f2236-5f45-462a-a582-2371b5d6d1b0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:53.769891 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f2236-5f45-462a-a582-2371b5d6d1b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:53.772667 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f2236-5f45-462a-a582-2371b5d6d1b0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:53.788146 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 511/1008] 10.222.0.76 () {56 vars in 1112 bytes} [Thu May 26 17:51:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:53.790147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d405f3-b9aa-4f95-9bce-3a1cd2ded7ec None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:53.911308 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d405f3-b9aa-4f95-9bce-3a1cd2ded7ec None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:53.947078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a4628-85fa-4802-95a9-0d9e8e38c41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:53.947751 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a4628-85fa-4802-95a9-0d9e8e38c41e None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:53.948034 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a4628-85fa-4802-95a9-0d9e8e38c41e None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:53.948308 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a4628-85fa-4802-95a9-0d9e8e38c41e None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:53.949018 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a4628-85fa-4802-95a9-0d9e8e38c41e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:53.949323 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a4628-85fa-4802-95a9-0d9e8e38c41e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:53.950724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a4628-85fa-4802-95a9-0d9e8e38c41e None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:53.961052 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 498/1009] 10.222.0.76 () {56 vars in 1112 bytes} [Thu May 26 17:51:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:53.964485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f2236-5f45-462a-a582-2371b5d6d1b0 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:54.046629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:54.074017 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f2236-5f45-462a-a582-2371b5d6d1b0 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:54.109595 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41815d8-09fe-42b9-b98e-e5eb52afab9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:54.110290 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41815d8-09fe-42b9-b98e-e5eb52afab9a None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:54.110578 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41815d8-09fe-42b9-b98e-e5eb52afab9a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:54.111059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41815d8-09fe-42b9-b98e-e5eb52afab9a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:54.111935 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41815d8-09fe-42b9-b98e-e5eb52afab9a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:54.113338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41815d8-09fe-42b9-b98e-e5eb52afab9a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:54.142434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 512/1010] 10.222.0.76 () {60 vars in 1144 bytes} [Thu May 26 17:51:53 2022] POST /identity/v3/endpoints => generated 334 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:51:54.822099 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86784175-0740-46a8-afc9-28226556be60 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:54.822743 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86784175-0740-46a8-afc9-28226556be60 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:54.822743 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86784175-0740-46a8-afc9-28226556be60 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:54.824228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 499/1011] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:54.829463 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570dbf4-a0ce-4007-b23e-243100a80e1d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:54.830509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570dbf4-a0ce-4007-b23e-243100a80e1d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:54.830734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570dbf4-a0ce-4007-b23e-243100a80e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:54.848767 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8570dbf4-a0ce-4007-b23e-243100a80e1d None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:54.849973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8570dbf4-a0ce-4007-b23e-243100a80e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:54.934511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 513/1012] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:54 2022] POST /identity/v3/auth/tokens => generated 1812 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:54.952621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86784175-0740-46a8-afc9-28226556be60 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:55.030147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:55.074533 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86784175-0740-46a8-afc9-28226556be60 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:55.109554 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46683355-8e87-44d1-bc38-ada0aed986d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:55.110417 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46683355-8e87-44d1-bc38-ada0aed986d9 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:55.111271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46683355-8e87-44d1-bc38-ada0aed986d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:55.111735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46683355-8e87-44d1-bc38-ada0aed986d9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:55.112424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46683355-8e87-44d1-bc38-ada0aed986d9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:55.113794 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46683355-8e87-44d1-bc38-ada0aed986d9 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:55.126400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-46683355-8e87-44d1-bc38-ada0aed986d9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:51:55.126400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:55.126400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:55.126400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:55.126400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:55.126400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:55.126400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:55.126400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:55.126400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:51:55.126400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:55.126400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:55.126400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:55.126400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:55.126400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:55.126400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:55.126400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:55.126400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:55.128727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:55.128727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:55.128727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:55.128727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:55.128727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:55.128727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:55.128727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:55.128727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:55.128727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:55.128727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:55.128727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:55.128727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:55.128727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:55.128727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:55.128727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:55.128727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:55.129601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:55.129601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:55.129601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:55.129601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:55.129601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:55.129601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:55.129601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:55.129601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:55.129601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:55.129601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:55.129601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:55.129601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:55.129601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 17:51:55.129601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:55.137770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 500/1013] 10.222.0.76 () {56 vars in 1126 bytes} [Thu May 26 17:51:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:55.138006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570dbf4-a0ce-4007-b23e-243100a80e1d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:55.201170 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:55.256047 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570dbf4-a0ce-4007-b23e-243100a80e1d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:55.295243 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd60dd-f407-44ab-b2b8-f03993d15758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:55.295910 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd60dd-f407-44ab-b2b8-f03993d15758 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:55.296193 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd60dd-f407-44ab-b2b8-f03993d15758 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:55.296468 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd60dd-f407-44ab-b2b8-f03993d15758 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:55.297168 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd60dd-f407-44ab-b2b8-f03993d15758 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:55.297476 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd60dd-f407-44ab-b2b8-f03993d15758 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:55.300103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd60dd-f407-44ab-b2b8-f03993d15758 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:55.310090 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 514/1014] 10.222.0.76 () {56 vars in 1122 bytes} [Thu May 26 17:51:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:55.311578 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46683355-8e87-44d1-bc38-ada0aed986d9 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:55.422806 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46683355-8e87-44d1-bc38-ada0aed986d9 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:55.452490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:55.462166 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12868fea-bfec-4da9-9eec-7e5e6f24f631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:55.462857 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12868fea-bfec-4da9-9eec-7e5e6f24f631 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:55.463141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12868fea-bfec-4da9-9eec-7e5e6f24f631 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:55.463415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12868fea-bfec-4da9-9eec-7e5e6f24f631 None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:55.464126 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12868fea-bfec-4da9-9eec-7e5e6f24f631 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:55.464436 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12868fea-bfec-4da9-9eec-7e5e6f24f631 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:55.465803 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12868fea-bfec-4da9-9eec-7e5e6f24f631 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:55.476679 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 501/1015] 10.222.0.76 () {56 vars in 1122 bytes} [Thu May 26 17:51:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:55.477726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd60dd-f407-44ab-b2b8-f03993d15758 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:55.597118 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd60dd-f407-44ab-b2b8-f03993d15758 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:55.618626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:55.634572 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc93b3-d77a-4c84-a0a6-92c06bf057b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:55.635237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc93b3-d77a-4c84-a0a6-92c06bf057b7 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:55.635626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc93b3-d77a-4c84-a0a6-92c06bf057b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:55.635898 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc93b3-d77a-4c84-a0a6-92c06bf057b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:55.636676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc93b3-d77a-4c84-a0a6-92c06bf057b7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:55.638059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc93b3-d77a-4c84-a0a6-92c06bf057b7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:55.655618 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 515/1016] 10.222.0.76 () {60 vars in 1144 bytes} [Thu May 26 17:51:55 2022] POST /identity/v3/endpoints => generated 325 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:51:56.092080 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae58d4-face-47f5-96eb-32c6d6067bbd None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:56.092660 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae58d4-face-47f5-96eb-32c6d6067bbd None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:56.092660 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae58d4-face-47f5-96eb-32c6d6067bbd None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:56.094158 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 502/1017] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:56.099687 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed4a02-4c20-4332-9bd9-e6d24728ceac None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:56.100174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed4a02-4c20-4332-9bd9-e6d24728ceac None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:56.100289 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed4a02-4c20-4332-9bd9-e6d24728ceac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:56.117034 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5bed4a02-4c20-4332-9bd9-e6d24728ceac None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:56.118248 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5bed4a02-4c20-4332-9bd9-e6d24728ceac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:56.203521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 516/1018] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:56.226331 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae58d4-face-47f5-96eb-32c6d6067bbd None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:56.337849 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae58d4-face-47f5-96eb-32c6d6067bbd None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:56.374367 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2bd0fa-5cfa-4055-bac2-46553a37621d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:56.375091 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bd0fa-5cfa-4055-bac2-46553a37621d None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:56.375373 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bd0fa-5cfa-4055-bac2-46553a37621d None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:56.375650 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bd0fa-5cfa-4055-bac2-46553a37621d None admin] PATH_INFO: `/v3/services/image` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:56.376865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2bd0fa-5cfa-4055-bac2-46553a37621d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:56.378277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2bd0fa-5cfa-4055-bac2-46553a37621d None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:56.390391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-0e2bd0fa-5cfa-4055-bac2-46553a37621d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 17:51:56.390391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:56.390391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:56.390391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:56.390391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:56.390391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:56.390391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:56.390391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:56.390391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 17:51:56.390391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:56.390391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:56.390391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:56.390391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:56.390391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:56.390391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:56.390391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:56.390391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:56.391474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:56.391474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:56.391474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:56.391474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:56.391474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:56.391474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:56.391474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:56.391474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:56.391474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:56.391474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:56.391474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:56.391474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:56.391474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:56.391474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:56.391474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:56.391474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:56.392503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:56.392503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:56.392503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:56.392503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:56.392503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:56.392503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:56.392503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:56.392503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:56.392503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:56.392503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:56.392503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:56.392503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:56.392503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 17:51:56.392503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:56.394398 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 503/1019] 10.222.0.76 () {56 vars in 1105 bytes} [Thu May 26 17:51:56 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:56.398228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed4a02-4c20-4332-9bd9-e6d24728ceac None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:56.504851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed4a02-4c20-4332-9bd9-e6d24728ceac None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:56.508885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482dbcd-5506-403b-a228-38f969711ed7 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:56.509174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482dbcd-5506-403b-a228-38f969711ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:56.509455 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482dbcd-5506-403b-a228-38f969711ed7 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:56.511009 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 504/1020] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:56.513412 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:56.516424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bc37c-45af-4fdd-b7e8-262247cf22ca None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:56.516729 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bc37c-45af-4fdd-b7e8-262247cf22ca None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:56.517119 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bc37c-45af-4fdd-b7e8-262247cf22ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:56.535999 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-c72bc37c-45af-4fdd-b7e8-262247cf22ca None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:56.537286 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-c72bc37c-45af-4fdd-b7e8-262247cf22ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:56.551680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09beac3c-a760-4155-bc6b-cf101e0a5183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:56.552371 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09beac3c-a760-4155-bc6b-cf101e0a5183 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:56.552652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09beac3c-a760-4155-bc6b-cf101e0a5183 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:56.553038 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09beac3c-a760-4155-bc6b-cf101e0a5183 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:56.553647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09beac3c-a760-4155-bc6b-cf101e0a5183 None admin] RBAC: Adding query filter params (name=image) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:56.553959 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09beac3c-a760-4155-bc6b-cf101e0a5183 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:56.555431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09beac3c-a760-4155-bc6b-cf101e0a5183 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:56.568220 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 517/1021] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:56.570223 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09beac3c-a760-4155-bc6b-cf101e0a5183 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:56.606681 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:56.632264 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 505/1022] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:51:56.655713 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72bc37c-45af-4fdd-b7e8-262247cf22ca None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:56.693042 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09beac3c-a760-4155-bc6b-cf101e0a5183 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:56.729759 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea05830-9cae-4ef4-a744-1afc70cd581b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:56.730824 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea05830-9cae-4ef4-a744-1afc70cd581b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:56.731108 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea05830-9cae-4ef4-a744-1afc70cd581b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:56.731389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea05830-9cae-4ef4-a744-1afc70cd581b None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:56.732361 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea05830-9cae-4ef4-a744-1afc70cd581b None admin] RBAC: Adding query filter params (type=image) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:56.732474 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea05830-9cae-4ef4-a744-1afc70cd581b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:56.733769 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea05830-9cae-4ef4-a744-1afc70cd581b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:56.747209 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 518/1023] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:56.748104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea05830-9cae-4ef4-a744-1afc70cd581b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:56.772072 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72bc37c-45af-4fdd-b7e8-262247cf22ca None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:56.792916 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:56.820243 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca05d6d-ae3e-4480-8c81-83b5bd28bb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:56.821324 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca05d6d-ae3e-4480-8c81-83b5bd28bb6c None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:56.821786 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca05d6d-ae3e-4480-8c81-83b5bd28bb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:56.822250 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca05d6d-ae3e-4480-8c81-83b5bd28bb6c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:56.836222 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca05d6d-ae3e-4480-8c81-83b5bd28bb6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:51:56.846805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-aca05d6d-ae3e-4480-8c81-83b5bd28bb6c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:51:56.846805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:56.846805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:56.846805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:56.846805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:56.846805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:56.846805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:56.846805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:56.846805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:56.846805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:56.846805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:56.846805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:56.846805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:51:56.846805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:51:56.846805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:51:56.846805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:51:56.848122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:51:56.848122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:51:56.848122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:51:56.848122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:51:56.848122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:56.848122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:56.848122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:56.848122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:56.848122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:56.848122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:56.848122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:56.848122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:56.848122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:56.848122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:56.848122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:56.848122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:56.849168 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:56.849168 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:56.849168 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:56.849168 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:56.849168 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:51:56.849168 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:51:56.849168 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:51:56.849168 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:51:56.849168 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:51:56.849168 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:51:56.849168 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:51:56.849168 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:56.856660 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 506/1024] 10.222.0.76 () {56 vars in 1120 bytes} [Thu May 26 17:51:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:51:56.857770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca05d6d-ae3e-4480-8c81-83b5bd28bb6c None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:56.895794 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea05830-9cae-4ef4-a744-1afc70cd581b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:56.932427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:56.932621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae629c-2535-4ddb-a05b-5c55a9874d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:56.933226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae629c-2535-4ddb-a05b-5c55a9874d40 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:56.933512 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae629c-2535-4ddb-a05b-5c55a9874d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:56.933789 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae629c-2535-4ddb-a05b-5c55a9874d40 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:56.934630 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae629c-2535-4ddb-a05b-5c55a9874d40 None admin] RBAC: Adding query filter params (interface=public, service_id=be97ecd27d4245bcab1dace854af159c, region_id=RegionOne) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:56.935307 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae629c-2535-4ddb-a05b-5c55a9874d40 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:56.936317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae629c-2535-4ddb-a05b-5c55a9874d40 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:56.946746 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 519/1025] 10.222.0.76 () {56 vars in 1251 bytes} [Thu May 26 17:51:56 2022] GET /identity/v3/endpoints?service_id=be97ecd27d4245bcab1dace854af159c&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:56.947876 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae629c-2535-4ddb-a05b-5c55a9874d40 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:57.028883 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca05d6d-ae3e-4480-8c81-83b5bd28bb6c None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:57.072960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5077ca-10b2-4c9c-9776-74c3e8f6976d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:57.073640 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5077ca-10b2-4c9c-9776-74c3e8f6976d None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:57.073925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5077ca-10b2-4c9c-9776-74c3e8f6976d None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:57.074481 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:57.075091 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5077ca-10b2-4c9c-9776-74c3e8f6976d None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:57.075892 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5077ca-10b2-4c9c-9776-74c3e8f6976d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:57.076496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5077ca-10b2-4c9c-9776-74c3e8f6976d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:57.078646 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5077ca-10b2-4c9c-9776-74c3e8f6976d None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:57.081026 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae629c-2535-4ddb-a05b-5c55a9874d40 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:57.089291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 507/1026] 10.222.0.76 () {56 vars in 1115 bytes} [Thu May 26 17:51:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:57.095396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5077ca-10b2-4c9c-9776-74c3e8f6976d None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:57.125574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d68906-9845-4eeb-a86d-ca503bc11d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:57.127234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d68906-9845-4eeb-a86d-ca503bc11d2f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:57.127522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d68906-9845-4eeb-a86d-ca503bc11d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:57.128060 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d68906-9845-4eeb-a86d-ca503bc11d2f None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:57.129207 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d68906-9845-4eeb-a86d-ca503bc11d2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:57.131139 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d68906-9845-4eeb-a86d-ca503bc11d2f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:57.140768 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 520/1027] 10.222.0.76 () {56 vars in 1087 bytes} [Thu May 26 17:51:56 2022] GET /identity/v3/services => generated 1957 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 17:51:57.239840 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5077ca-10b2-4c9c-9776-74c3e8f6976d None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:57.251457 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:57.286449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa2ac8-d881-4c6c-8f42-016b4a810979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:57.287287 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa2ac8-d881-4c6c-8f42-016b4a810979 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:57.287632 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa2ac8-d881-4c6c-8f42-016b4a810979 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:57.287911 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa2ac8-d881-4c6c-8f42-016b4a810979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:57.301165 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa2ac8-d881-4c6c-8f42-016b4a810979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:57.302693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa2ac8-d881-4c6c-8f42-016b4a810979 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:57.322148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-02aa2ac8-d881-4c6c-8f42-016b4a810979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:57.322148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:57.322148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:57.322148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:57.322148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:57.322148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:57.322148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:57.322148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:57.322148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:57.322148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:57.322148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:57.322148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:57.322148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:57.322148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:57.322148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:57.323234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:57.323234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:57.323234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:57.323234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:57.323234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:57.323234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:57.323234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:57.323234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:57.323234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:57.323234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:57.323234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:57.323234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:57.323234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:57.323234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:57.323234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:57.323234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:57.323234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:57.324155 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:57.324155 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:57.324155 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:57.324155 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:57.326614 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 508/1028] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:51:57.329895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d68906-9845-4eeb-a86d-ca503bc11d2f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:57.445287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d68906-9845-4eeb-a86d-ca503bc11d2f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:57.482809 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b014a6e6-027c-4f25-9557-83045cc06c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:57.483482 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014a6e6-027c-4f25-9557-83045cc06c0f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:57.483772 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014a6e6-027c-4f25-9557-83045cc06c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:57.484050 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014a6e6-027c-4f25-9557-83045cc06c0f None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:57.484773 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b014a6e6-027c-4f25-9557-83045cc06c0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:57.485082 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b014a6e6-027c-4f25-9557-83045cc06c0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:57.486518 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b014a6e6-027c-4f25-9557-83045cc06c0f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:57.499617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 521/1029] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:57.505698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa2ac8-d881-4c6c-8f42-016b4a810979 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:57.615601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa2ac8-d881-4c6c-8f42-016b4a810979 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:57.650649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff7bf2-6db0-4c49-8f87-63452fdd99e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:57.651449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff7bf2-6db0-4c49-8f87-63452fdd99e1 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:57.651735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff7bf2-6db0-4c49-8f87-63452fdd99e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:57.652014 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff7bf2-6db0-4c49-8f87-63452fdd99e1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:57.661001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff7bf2-6db0-4c49-8f87-63452fdd99e1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:57.662542 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff7bf2-6db0-4c49-8f87-63452fdd99e1 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:57.677617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-e1ff7bf2-6db0-4c49-8f87-63452fdd99e1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 17:51:57.677617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:57.677617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:57.677617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:57.677617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:57.677617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:57.677617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:57.677617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:57.677617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:57.677617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:57.677617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:57.677617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:57.677617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:51:57.677617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:51:57.677617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:51:57.677617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:51:57.678744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:57.678744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:57.678744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:51:57.678744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:57.678744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:51:57.678744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:51:57.678744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:57.678744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:57.678744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:57.678744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:57.678744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:57.678744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:57.678744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:57.678744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:57.678744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:57.678744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:57.678744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:57.678744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:57.679744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:57.679744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:57.679744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:51:57.679744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:51:57.679744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:51:57.679744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:51:57.679744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:51:57.679744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:51:57.679744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 17:51:57.679744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:57.682277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 509/1030] 10.222.0.76 () {56 vars in 1093 bytes} [Thu May 26 17:51:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:57.684716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b014a6e6-027c-4f25-9557-83045cc06c0f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:57.711862 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:57.798535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b014a6e6-027c-4f25-9557-83045cc06c0f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:57.840624 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd524d0-b1cf-41d6-a9d0-8adf855c050e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:57.841345 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd524d0-b1cf-41d6-a9d0-8adf855c050e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:57.841636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd524d0-b1cf-41d6-a9d0-8adf855c050e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:57.841921 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd524d0-b1cf-41d6-a9d0-8adf855c050e None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:57.842880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd524d0-b1cf-41d6-a9d0-8adf855c050e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:57.843199 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd524d0-b1cf-41d6-a9d0-8adf855c050e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:57.844796 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd524d0-b1cf-41d6-a9d0-8adf855c050e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:57.870434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 522/1031] 10.222.0.76 () {56 vars in 1133 bytes} [Thu May 26 17:51:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:57.874988 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff7bf2-6db0-4c49-8f87-63452fdd99e1 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:57.904611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:57.997584 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff7bf2-6db0-4c49-8f87-63452fdd99e1 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:58.034818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743bae75-82cf-4231-b983-92fcacca7111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:58.035505 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bae75-82cf-4231-b983-92fcacca7111 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:58.036049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bae75-82cf-4231-b983-92fcacca7111 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:58.036337 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bae75-82cf-4231-b983-92fcacca7111 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:58.110319 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743bae75-82cf-4231-b983-92fcacca7111 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:58.112301 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743bae75-82cf-4231-b983-92fcacca7111 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:58.142783 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:58.237942 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e1e8e-d3a4-4302-a543-c265692a325c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:58.238542 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e1e8e-d3a4-4302-a543-c265692a325c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:58.238542 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e1e8e-d3a4-4302-a543-c265692a325c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:58.239894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 523/1032] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:58.241770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 510/1033] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:51:57 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 17:51:58.245257 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e05a6-4acf-40df-b0a4-71395c0c465e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:58.245543 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e05a6-4acf-40df-b0a4-71395c0c465e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:58.245823 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e05a6-4acf-40df-b0a4-71395c0c465e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:58.247695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743bae75-82cf-4231-b983-92fcacca7111 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:58.264283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3b6e05a6-4acf-40df-b0a4-71395c0c465e None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:58.265178 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3b6e05a6-4acf-40df-b0a4-71395c0c465e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:58.346838 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 524/1034] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:58.369341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743bae75-82cf-4231-b983-92fcacca7111 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:58.370223 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e05a6-4acf-40df-b0a4-71395c0c465e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:58.381880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:58.399434 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:58.422846 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382e35d-3182-48e5-8886-e0848de03280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:58.424237 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382e35d-3182-48e5-8886-e0848de03280 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:58.424520 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382e35d-3182-48e5-8886-e0848de03280 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:58.424796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382e35d-3182-48e5-8886-e0848de03280 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:58.437655 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382e35d-3182-48e5-8886-e0848de03280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:58.439425 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382e35d-3182-48e5-8886-e0848de03280 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:58.462460 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-1382e35d-3182-48e5-8886-e0848de03280 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:58.462460 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:58.462460 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:58.462460 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:58.462460 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:58.462460 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:58.462460 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:58.462460 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:58.462460 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:58.462460 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:58.462460 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:58.462460 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:58.462460 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:51:58.462460 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:51:58.462460 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:51:58.463501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:51:58.463501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:58.463501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:58.463501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:58.463501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:58.463501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:58.463501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:58.463501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:58.463501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:58.463501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:58.463501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:58.463501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:58.463501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:58.463501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:58.463501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:58.463501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:58.463501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:58.469464 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:51:58.469464 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:51:58.469464 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:51:58.469464 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:58.469464 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 511/1035] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:51:58.470767 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382e35d-3182-48e5-8886-e0848de03280 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:58.488800 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e05a6-4acf-40df-b0a4-71395c0c465e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:58.538002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23678948-adf0-4c02-a9ba-213e05495bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:58.539361 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23678948-adf0-4c02-a9ba-213e05495bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:58.539880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23678948-adf0-4c02-a9ba-213e05495bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:58.539880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23678948-adf0-4c02-a9ba-213e05495bb8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:58.541183 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23678948-adf0-4c02-a9ba-213e05495bb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:58.542628 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23678948-adf0-4c02-a9ba-213e05495bb8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:58.544242 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 525/1036] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:58.551201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23678948-adf0-4c02-a9ba-213e05495bb8 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:58.601110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382e35d-3182-48e5-8886-e0848de03280 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:58.647787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c86973-d15c-4906-a4ab-945f7bf6b562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:58.648312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c86973-d15c-4906-a4ab-945f7bf6b562 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:58.649136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c86973-d15c-4906-a4ab-945f7bf6b562 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:58.649136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c86973-d15c-4906-a4ab-945f7bf6b562 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:58.654970 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c86973-d15c-4906-a4ab-945f7bf6b562 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:58.654970 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c86973-d15c-4906-a4ab-945f7bf6b562 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:58.654970 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c86973-d15c-4906-a4ab-945f7bf6b562 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:58.663617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:58.666407 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 512/1037] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:58.671792 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c86973-d15c-4906-a4ab-945f7bf6b562 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:58.701998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23678948-adf0-4c02-a9ba-213e05495bb8 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:58.707812 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:58.752781 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9cbe3-9bb9-4929-a76a-af29c1ce7736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:58.753487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9cbe3-9bb9-4929-a76a-af29c1ce7736 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:58.753774 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9cbe3-9bb9-4929-a76a-af29c1ce7736 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:58.754047 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9cbe3-9bb9-4929-a76a-af29c1ce7736 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:58.754919 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9cbe3-9bb9-4929-a76a-af29c1ce7736 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:58.756475 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9cbe3-9bb9-4929-a76a-af29c1ce7736 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:58.794643 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 526/1038] 10.222.0.76 () {60 vars in 1141 bytes} [Thu May 26 17:51:58 2022] POST /identity/v3/projects => generated 311 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:51:58.794821 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c86973-d15c-4906-a4ab-945f7bf6b562 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:58.841298 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317cd336-0eba-48da-bc9f-27c7415c076e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:58.842114 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317cd336-0eba-48da-bc9f-27c7415c076e None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:58.842433 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317cd336-0eba-48da-bc9f-27c7415c076e None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:58.842712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317cd336-0eba-48da-bc9f-27c7415c076e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:58.851615 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317cd336-0eba-48da-bc9f-27c7415c076e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:58.856094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317cd336-0eba-48da-bc9f-27c7415c076e None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:58.876043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-317cd336-0eba-48da-bc9f-27c7415c076e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:58.876043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:58.876043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:58.876043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:58.876043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:58.876043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:58.876043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:58.876043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:58.876043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:58.876043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:58.876043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:58.876043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:58.876043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:51:58.876043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:51:58.876043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:51:58.877085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:51:58.877085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:58.877085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:58.877085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:58.877085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:58.877085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:58.877085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:58.877085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:58.877085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:51:58.877085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:58.877085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:58.877085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:58.877085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:58.877085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:58.877085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:58.877085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:58.877085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:58.883225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:51:58.883225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:51:58.883225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:51:58.883225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:51:58.883225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:51:58.883225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:51:58.883225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:51:58.883225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:51:58.883225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 513/1039] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:51:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:51:58.883747 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9cbe3-9bb9-4929-a76a-af29c1ce7736 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:59.017984 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9cbe3-9bb9-4929-a76a-af29c1ce7736 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:59.051198 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:59.058510 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c341db-d086-43ff-be4a-344d0c771c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:59.059261 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c341db-d086-43ff-be4a-344d0c771c68 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:59.059795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c341db-d086-43ff-be4a-344d0c771c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:59.060287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c341db-d086-43ff-be4a-344d0c771c68 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:59.061428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c341db-d086-43ff-be4a-344d0c771c68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:59.061735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c341db-d086-43ff-be4a-344d0c771c68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:59.063233 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c341db-d086-43ff-be4a-344d0c771c68 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:59.077974 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 527/1040] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:51:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:51:59.085087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317cd336-0eba-48da-bc9f-27c7415c076e None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:59.206337 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:51:59.218991 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317cd336-0eba-48da-bc9f-27c7415c076e None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:59.265996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157bfb87-b910-4608-9518-1b62de7f5c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:59.266758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bfb87-b910-4608-9518-1b62de7f5c4f None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:59.267857 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bfb87-b910-4608-9518-1b62de7f5c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:59.268635 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bfb87-b910-4608-9518-1b62de7f5c4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:59.269477 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157bfb87-b910-4608-9518-1b62de7f5c4f None admin] RBAC: Adding query filter params (role.id=04c312d9dc3249fe8fbff78befef65f9, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=1899c317083d40f5b490cd9eb63fe521) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:51:59.269788 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157bfb87-b910-4608-9518-1b62de7f5c4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:59.271314 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157bfb87-b910-4608-9518-1b62de7f5c4f None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:59.282234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 514/1041] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:51:59 2022] GET /identity/v3/role_assignments?user.id=1899c317083d40f5b490cd9eb63fe521&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=04c312d9dc3249fe8fbff78befef65f9 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:51:59.635196 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207e5e5-4b8c-4e90-8b5d-5330a51be877 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:59.636006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207e5e5-4b8c-4e90-8b5d-5330a51be877 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:59.636196 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207e5e5-4b8c-4e90-8b5d-5330a51be877 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:59.637821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 528/1042] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:51:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:51:59.643850 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23b58c-398a-4f60-956c-41f0e74ef4ba None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:59.644117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23b58c-398a-4f60-956c-41f0e74ef4ba None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:59.644404 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23b58c-398a-4f60-956c-41f0e74ef4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:59.661939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-5d23b58c-398a-4f60-956c-41f0e74ef4ba None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:51:59.663183 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-5d23b58c-398a-4f60-956c-41f0e74ef4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:51:59.732544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 515/1043] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:51:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:51:59.756073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207e5e5-4b8c-4e90-8b5d-5330a51be877 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:51:59.873534 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207e5e5-4b8c-4e90-8b5d-5330a51be877 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:51:59.908988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c3811-0d1b-47dc-8eb6-18fd409b0d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:51:59.909698 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c3811-0d1b-47dc-8eb6-18fd409b0d99 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:51:59.909988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c3811-0d1b-47dc-8eb6-18fd409b0d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:51:59.910296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c3811-0d1b-47dc-8eb6-18fd409b0d99 None admin] PATH_INFO: `/v3/services/image` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:51:59.911216 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c3811-0d1b-47dc-8eb6-18fd409b0d99 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:51:59.912591 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c3811-0d1b-47dc-8eb6-18fd409b0d99 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:51:59.925208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-677c3811-0d1b-47dc-8eb6-18fd409b0d99 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 17:51:59.925208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:59.925208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:51:59.925208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:51:59.925208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:51:59.925208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:51:59.925208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:51:59.925208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:59.925208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 17:51:59.925208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:59.925208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:51:59.925208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:59.925208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:51:59.925208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:51:59.925208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:51:59.925208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:51:59.925208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:51:59.926420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:51:59.926420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:51:59.926420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:51:59.926420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:51:59.926420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:51:59.926420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:51:59.926420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:51:59.926420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:51:59.926420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:51:59.926420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:51:59.926420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:51:59.926420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:51:59.926420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:51:59.926420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:51:59.926420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:51:59.926420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:51:59.933970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:51:59.933970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:51:59.933970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:51:59.933970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:51:59.933970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:51:59.933970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:51:59.933970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:51:59.933970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:51:59.933970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:51:59.933970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:51:59.933970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:51:59.933970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:51:59.933970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 17:51:59.933970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:51:59.933970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 529/1044] 10.222.0.76 () {56 vars in 1105 bytes} [Thu May 26 17:51:59 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:51:59.937419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23b58c-398a-4f60-956c-41f0e74ef4ba None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:00.057267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:00.076580 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23b58c-398a-4f60-956c-41f0e74ef4ba None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:00.116848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6812e8-2c26-4d33-8169-b2325717a564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:00.117516 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6812e8-2c26-4d33-8169-b2325717a564 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:00.117799 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6812e8-2c26-4d33-8169-b2325717a564 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:00.118073 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6812e8-2c26-4d33-8169-b2325717a564 None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:00.118814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6812e8-2c26-4d33-8169-b2325717a564 None admin] RBAC: Adding query filter params (name=image) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:00.119119 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6812e8-2c26-4d33-8169-b2325717a564 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:00.120482 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6812e8-2c26-4d33-8169-b2325717a564 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:00.133141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 516/1045] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:51:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:00.133330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c3811-0d1b-47dc-8eb6-18fd409b0d99 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:00.197548 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:00.250385 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c3811-0d1b-47dc-8eb6-18fd409b0d99 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:00.285153 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27125bf-1b10-4478-8995-00556d9d8dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:00.286074 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27125bf-1b10-4478-8995-00556d9d8dbc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:00.286387 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27125bf-1b10-4478-8995-00556d9d8dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:00.286663 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27125bf-1b10-4478-8995-00556d9d8dbc None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:00.287365 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27125bf-1b10-4478-8995-00556d9d8dbc None admin] RBAC: Adding query filter params (type=image) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:00.287672 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27125bf-1b10-4478-8995-00556d9d8dbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:00.289061 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27125bf-1b10-4478-8995-00556d9d8dbc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:00.299216 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 530/1046] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:52:00 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:52:00.300869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6812e8-2c26-4d33-8169-b2325717a564 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:00.409740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6812e8-2c26-4d33-8169-b2325717a564 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:00.445608 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a2459-2ba2-4fd6-beda-452970d7af55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:00.446318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a2459-2ba2-4fd6-beda-452970d7af55 None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:00.446607 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a2459-2ba2-4fd6-beda-452970d7af55 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:00.446887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a2459-2ba2-4fd6-beda-452970d7af55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:00.447697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a2459-2ba2-4fd6-beda-452970d7af55 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:00.449025 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a2459-2ba2-4fd6-beda-452970d7af55 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:00.468187 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 517/1047] 10.222.0.76 () {60 vars in 1144 bytes} [Thu May 26 17:52:00 2022] POST /identity/v3/endpoints => generated 327 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 17:52:01.101686 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfce33-6bfb-4603-9a15-dce6db456850 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:01.102241 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfce33-6bfb-4603-9a15-dce6db456850 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:01.102353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfce33-6bfb-4603-9a15-dce6db456850 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:01.103696 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 531/1048] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:52:01.109066 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4fa45-59ba-44fc-9ab6-142eccebaf45 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:01.109356 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4fa45-59ba-44fc-9ab6-142eccebaf45 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:01.109637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4fa45-59ba-44fc-9ab6-142eccebaf45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:01.122548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:01.130264 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-58e4fa45-59ba-44fc-9ab6-142eccebaf45 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:01.130855 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-58e4fa45-59ba-44fc-9ab6-142eccebaf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:01.212809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 518/1049] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:52:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:52:01.238387 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfce33-6bfb-4603-9a15-dce6db456850 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:01.347399 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfce33-6bfb-4603-9a15-dce6db456850 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:01.367442 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:01.385487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fbe519-0c6f-41bb-99c8-5bd6a6cfdfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:01.386325 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fbe519-0c6f-41bb-99c8-5bd6a6cfdfda None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:01.386610 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fbe519-0c6f-41bb-99c8-5bd6a6cfdfda None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:01.386886 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fbe519-0c6f-41bb-99c8-5bd6a6cfdfda None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:01.388076 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fbe519-0c6f-41bb-99c8-5bd6a6cfdfda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:01.389536 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fbe519-0c6f-41bb-99c8-5bd6a6cfdfda None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:01.394591 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 532/1050] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:52:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:01.396956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4fa45-59ba-44fc-9ab6-142eccebaf45 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:01.503009 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4fa45-59ba-44fc-9ab6-142eccebaf45 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:01.524467 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4884d2-c4c4-4f4f-a722-ef08bf3a24bd None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:01.524816 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4884d2-c4c4-4f4f-a722-ef08bf3a24bd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:01.525034 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4884d2-c4c4-4f4f-a722-ef08bf3a24bd None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:01.526690 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 533/1051] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:52:01.532291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52124d93-581d-42c4-aede-967623db28dd None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:01.533244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52124d93-581d-42c4-aede-967623db28dd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:01.533406 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52124d93-581d-42c4-aede-967623db28dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:01.538394 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:01.546232 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0972d3-b82a-4fb3-823b-2f2dc1e3e56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:01.546954 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0972d3-b82a-4fb3-823b-2f2dc1e3e56e None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:01.547245 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0972d3-b82a-4fb3-823b-2f2dc1e3e56e None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:01.547524 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0972d3-b82a-4fb3-823b-2f2dc1e3e56e None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:01.548851 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0972d3-b82a-4fb3-823b-2f2dc1e3e56e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:01.550532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0972d3-b82a-4fb3-823b-2f2dc1e3e56e None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:01.552329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-52124d93-581d-42c4-aede-967623db28dd None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:01.553599 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-52124d93-581d-42c4-aede-967623db28dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:01.578223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 519/1052] 10.222.0.76 () {60 vars in 1132 bytes} [Thu May 26 17:52:01 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:52:01.645581 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 534/1053] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:52:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:52:01.669530 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0972d3-b82a-4fb3-823b-2f2dc1e3e56e None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:01.783510 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0972d3-b82a-4fb3-823b-2f2dc1e3e56e None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:01.824266 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c5f88-b2a1-46cf-8e9d-0a3c5a09d33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:01.825088 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c5f88-b2a1-46cf-8e9d-0a3c5a09d33d None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:01.825370 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c5f88-b2a1-46cf-8e9d-0a3c5a09d33d None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:01.825676 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c5f88-b2a1-46cf-8e9d-0a3c5a09d33d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:01.836945 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c5f88-b2a1-46cf-8e9d-0a3c5a09d33d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:52:01.845493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-4b7c5f88-b2a1-46cf-8e9d-0a3c5a09d33d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:52:01.845493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:52:01.845493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:52:01.845493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:52:01.845493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:52:01.845493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:52:01.845493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:52:01.845493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:52:01.845493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:52:01.845493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:52:01.845493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:52:01.845493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:52:01.845493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:52:01.845493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:52:01.845493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:52:01.845493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:52:01.846796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:52:01.846796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:52:01.846796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:52:01.846796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:52:01.846796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:52:01.846796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:52:01.846796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:52:01.846796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:52:01.846796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:52:01.846796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:52:01.846796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:52:01.846796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:52:01.846796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:52:01.846796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:52:01.846796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:52:01.846796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:52:01.847672 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:52:01.847672 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:52:01.847672 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:52:01.847672 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:52:01.847672 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:52:01.847672 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:52:01.847672 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:52:01.847672 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:52:01.847672 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:52:01.847672 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:52:01.847672 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:52:01.847672 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:52:01.847672 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 520/1054] 10.222.0.76 () {56 vars in 1120 bytes} [Thu May 26 17:52:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:52:01.853052 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52124d93-581d-42c4-aede-967623db28dd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:01.959142 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52124d93-581d-42c4-aede-967623db28dd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:01.977812 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:02.000026 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2851cb-229f-4f16-82ea-6843ec3c14d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:02.001079 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2851cb-229f-4f16-82ea-6843ec3c14d6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:02.001682 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2851cb-229f-4f16-82ea-6843ec3c14d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:02.002149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2851cb-229f-4f16-82ea-6843ec3c14d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:02.003283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2851cb-229f-4f16-82ea-6843ec3c14d6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:02.003805 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2851cb-229f-4f16-82ea-6843ec3c14d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:02.005438 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2851cb-229f-4f16-82ea-6843ec3c14d6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:02.015737 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 535/1055] 10.222.0.76 () {56 vars in 1115 bytes} [Thu May 26 17:52:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:02.021427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c5f88-b2a1-46cf-8e9d-0a3c5a09d33d None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:02.077043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:02.136305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c5f88-b2a1-46cf-8e9d-0a3c5a09d33d None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:02.178318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb29a4-5737-45bb-9dd7-81de1f1a92b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:02.179427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb29a4-5737-45bb-9dd7-81de1f1a92b0 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:02.179893 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb29a4-5737-45bb-9dd7-81de1f1a92b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:02.180351 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb29a4-5737-45bb-9dd7-81de1f1a92b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:02.192068 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb29a4-5737-45bb-9dd7-81de1f1a92b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:02.193842 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb29a4-5737-45bb-9dd7-81de1f1a92b0 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:02.211547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-27cb29a4-5737-45bb-9dd7-81de1f1a92b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:52:02.211547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:52:02.211547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:52:02.211547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:52:02.211547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:52:02.211547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:52:02.211547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:52:02.211547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:52:02.211547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:52:02.211547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:52:02.211547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:52:02.211547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:52:02.211547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:52:02.211547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:52:02.211547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:52:02.212474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:52:02.212474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:52:02.212474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:52:02.212474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:52:02.212474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:52:02.212474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:52:02.212474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:52:02.212474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:52:02.212474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:52:02.212474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:52:02.212474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:52:02.212474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:52:02.212474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:52:02.212474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:52:02.212474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:52:02.212474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:52:02.212474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:52:02.213438 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:52:02.213438 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:52:02.213438 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:52:02.213438 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:52:02.218337 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 521/1056] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:52:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:52:02.220380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2851cb-229f-4f16-82ea-6843ec3c14d6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:02.326313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2851cb-229f-4f16-82ea-6843ec3c14d6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:02.362298 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cbe77e-440d-4263-a1c9-c1150774ec44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:02.362969 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbe77e-440d-4263-a1c9-c1150774ec44 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:02.363257 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbe77e-440d-4263-a1c9-c1150774ec44 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:02.363536 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbe77e-440d-4263-a1c9-c1150774ec44 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:02.364260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cbe77e-440d-4263-a1c9-c1150774ec44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:02.364570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cbe77e-440d-4263-a1c9-c1150774ec44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:02.365954 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cbe77e-440d-4263-a1c9-c1150774ec44 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:02.382235 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 536/1057] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:02.384461 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb29a4-5737-45bb-9dd7-81de1f1a92b0 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:02.502149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:02.506267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb29a4-5737-45bb-9dd7-81de1f1a92b0 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:02.543003 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995adbfc-de4c-471c-876b-d971e0cdb1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:02.543822 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995adbfc-de4c-471c-876b-d971e0cdb1ff None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:02.544101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995adbfc-de4c-471c-876b-d971e0cdb1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:02.544372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995adbfc-de4c-471c-876b-d971e0cdb1ff None admin] PATH_INFO: `/v3/users/nova` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:02.552801 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995adbfc-de4c-471c-876b-d971e0cdb1ff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:02.554282 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995adbfc-de4c-471c-876b-d971e0cdb1ff None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:02.570037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-995adbfc-de4c-471c-876b-d971e0cdb1ff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 17:52:02.570037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:52:02.570037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:52:02.570037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:52:02.570037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:52:02.570037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:52:02.570037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:52:02.570037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:52:02.570037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:52:02.570037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:52:02.570037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:52:02.570037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:52:02.570037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:52:02.570037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:52:02.570037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:52:02.570037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:52:02.571213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:52:02.571213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:52:02.571213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:52:02.571213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:52:02.571213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:52:02.571213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:52:02.571213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:52:02.571213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:52:02.571213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:52:02.571213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:52:02.571213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:52:02.571213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:52:02.571213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:52:02.571213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:52:02.571213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:52:02.571213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:52:02.571213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:52:02.571213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:52:02.572215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:52:02.572215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:52:02.572215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:52:02.572215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:52:02.572215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:52:02.572215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:52:02.572215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:52:02.572215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:52:02.572215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 17:52:02.572215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:52:02.573722 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 522/1058] 10.222.0.76 () {56 vars in 1093 bytes} [Thu May 26 17:52:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:52:02.577638 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cbe77e-440d-4263-a1c9-c1150774ec44 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:02.624390 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:02.688262 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cbe77e-440d-4263-a1c9-c1150774ec44 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:02.725250 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43699627-e6f3-4604-82e3-50180aea914d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:02.725927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43699627-e6f3-4604-82e3-50180aea914d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:02.726236 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43699627-e6f3-4604-82e3-50180aea914d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:02.726514 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43699627-e6f3-4604-82e3-50180aea914d None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:02.727353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43699627-e6f3-4604-82e3-50180aea914d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:02.727667 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43699627-e6f3-4604-82e3-50180aea914d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:02.729099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43699627-e6f3-4604-82e3-50180aea914d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:02.752908 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 537/1059] 10.222.0.76 () {56 vars in 1133 bytes} [Thu May 26 17:52:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:02.754081 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995adbfc-de4c-471c-876b-d971e0cdb1ff None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:02.766775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f96e4-f256-4977-a58f-a1be7701b162 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:02.766997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f96e4-f256-4977-a58f-a1be7701b162 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:02.767275 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f96e4-f256-4977-a58f-a1be7701b162 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:02.768996 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 538/1060] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:52:02.774781 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b158aa-f060-4562-b06b-ef360971d075 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:02.775165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b158aa-f060-4562-b06b-ef360971d075 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:02.775273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b158aa-f060-4562-b06b-ef360971d075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:02.793829 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b9b158aa-f060-4562-b06b-ef360971d075 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:02.795084 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b9b158aa-f060-4562-b06b-ef360971d075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:02.866576 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 539/1061] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:52:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:52:02.877598 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995adbfc-de4c-471c-876b-d971e0cdb1ff None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:02.888348 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b158aa-f060-4562-b06b-ef360971d075 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:02.917728 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad34f3-e7af-47e1-bf49-cfca56d4390f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:02.918566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad34f3-e7af-47e1-bf49-cfca56d4390f None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:02.918901 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad34f3-e7af-47e1-bf49-cfca56d4390f None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:02.919177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad34f3-e7af-47e1-bf49-cfca56d4390f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:02.919548 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:02.932813 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad34f3-e7af-47e1-bf49-cfca56d4390f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:02.934304 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad34f3-e7af-47e1-bf49-cfca56d4390f None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:02.952058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-06ad34f3-e7af-47e1-bf49-cfca56d4390f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:52:02.952058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:52:02.952058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:52:02.952058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:52:02.952058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:52:02.952058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:52:02.952058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:52:02.952058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:52:02.952058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:52:02.952058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:52:02.952058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:52:02.952058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:52:02.952058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:52:02.952058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:52:02.952058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:52:02.953094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:52:02.953094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:52:02.953094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:52:02.953094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:52:02.953094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:52:02.953094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:52:02.953094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:52:02.953094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:52:02.953094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:52:02.953094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:52:02.953094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:52:02.953094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:52:02.953094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:52:02.953094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:52:02.953094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:52:02.953094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:52:02.953094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:52:02.954039 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:52:02.954039 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:52:02.954039 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:52:02.954039 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:52:02.954039 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 523/1062] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:52:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:52:02.964052 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad34f3-e7af-47e1-bf49-cfca56d4390f None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:02.980925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:03.012340 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b158aa-f060-4562-b06b-ef360971d075 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:03.053793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2943cff-7a57-4f26-b967-a7c2e50c31f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:03.054493 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2943cff-7a57-4f26-b967-a7c2e50c31f0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:03.054774 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2943cff-7a57-4f26-b967-a7c2e50c31f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:03.055233 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2943cff-7a57-4f26-b967-a7c2e50c31f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:03.086609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad34f3-e7af-47e1-bf49-cfca56d4390f None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:03.121056 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d647d1-1527-4836-a290-cca2b5f26a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:03.121724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d647d1-1527-4836-a290-cca2b5f26a7e None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:03.122011 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d647d1-1527-4836-a290-cca2b5f26a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:03.122321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d647d1-1527-4836-a290-cca2b5f26a7e None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:03.123043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d647d1-1527-4836-a290-cca2b5f26a7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:03.123354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d647d1-1527-4836-a290-cca2b5f26a7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:03.124730 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d647d1-1527-4836-a290-cca2b5f26a7e None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:03.128378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2943cff-7a57-4f26-b967-a7c2e50c31f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:03.130048 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2943cff-7a57-4f26-b967-a7c2e50c31f0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:03.150581 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 524/1063] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:52:03.152853 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d647d1-1527-4836-a290-cca2b5f26a7e None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:03.162298 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:03.232182 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:03.246361 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 540/1064] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:52:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 17:52:03.248944 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2943cff-7a57-4f26-b967-a7c2e50c31f0 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:03.269383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d647d1-1527-4836-a290-cca2b5f26a7e None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:03.307562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac98c8-a156-4621-9282-0e15090fd595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:03.308370 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac98c8-a156-4621-9282-0e15090fd595 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:03.308663 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac98c8-a156-4621-9282-0e15090fd595 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:03.308930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac98c8-a156-4621-9282-0e15090fd595 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:03.316841 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac98c8-a156-4621-9282-0e15090fd595 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:03.318326 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac98c8-a156-4621-9282-0e15090fd595 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:03.333303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-d5ac98c8-a156-4621-9282-0e15090fd595 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:52:03.333303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:52:03.333303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:52:03.333303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:52:03.333303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:52:03.333303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:52:03.333303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:52:03.333303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:52:03.333303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:52:03.333303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:52:03.333303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:52:03.333303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:52:03.333303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:52:03.333303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:52:03.333303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:52:03.334303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:52:03.334303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:52:03.334303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:52:03.334303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:52:03.334303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:52:03.334303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:52:03.334303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:52:03.334303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:52:03.334303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:52:03.334303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:52:03.334303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:52:03.334303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:52:03.334303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:52:03.334303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:52:03.334303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:52:03.334303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:52:03.334303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:52:03.335239 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:52:03.335239 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:52:03.335239 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:52:03.335239 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:52:03.335239 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:52:03.335239 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:52:03.335239 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:52:03.335239 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:52:03.335239 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 525/1065] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:52:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:52:03.340156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac98c8-a156-4621-9282-0e15090fd595 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:03.361856 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2943cff-7a57-4f26-b967-a7c2e50c31f0 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:03.391572 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:03.402238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a8a8b-7d29-487f-8fac-54dcf9a91340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:03.403039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a8a8b-7d29-487f-8fac-54dcf9a91340 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:03.403437 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a8a8b-7d29-487f-8fac-54dcf9a91340 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:03.403715 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a8a8b-7d29-487f-8fac-54dcf9a91340 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:03.415290 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a8a8b-7d29-487f-8fac-54dcf9a91340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:03.416901 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a8a8b-7d29-487f-8fac-54dcf9a91340 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:03.436607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-6a0a8a8b-7d29-487f-8fac-54dcf9a91340 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:52:03.436607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:52:03.436607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:52:03.436607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:52:03.436607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:52:03.436607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:52:03.436607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:52:03.436607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:52:03.436607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:52:03.436607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:52:03.436607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:52:03.436607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:52:03.436607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:52:03.436607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:52:03.436607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:52:03.437597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:52:03.437597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:52:03.437597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:52:03.437597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:52:03.437597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:52:03.437597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:52:03.437597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:52:03.437597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:52:03.437597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:52:03.437597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:52:03.437597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:52:03.437597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:52:03.437597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:52:03.437597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:52:03.437597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:52:03.437597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:52:03.437597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:52:03.438550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:52:03.438550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:52:03.438550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:52:03.438550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:52:03.438550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 541/1066] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:52:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:52:03.444917 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a8a8b-7d29-487f-8fac-54dcf9a91340 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:03.453183 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac98c8-a156-4621-9282-0e15090fd595 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:03.487696 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:03.494603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3b61d-f2c4-4532-8846-7d431df99efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:03.495387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3b61d-f2c4-4532-8846-7d431df99efc None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:03.495669 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3b61d-f2c4-4532-8846-7d431df99efc None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:03.496247 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3b61d-f2c4-4532-8846-7d431df99efc None admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:03.496852 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3b61d-f2c4-4532-8846-7d431df99efc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:03.497165 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3b61d-f2c4-4532-8846-7d431df99efc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:03.498810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3b61d-f2c4-4532-8846-7d431df99efc None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:03.519845 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 526/1067] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:52:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:03.522011 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3b61d-f2c4-4532-8846-7d431df99efc None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:03.554597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a8a8b-7d29-487f-8fac-54dcf9a91340 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:03.591665 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae4bba-17ae-48de-8c8d-c063825aaee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:03.592309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae4bba-17ae-48de-8c8d-c063825aaee8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:03.592595 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae4bba-17ae-48de-8c8d-c063825aaee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:03.592912 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae4bba-17ae-48de-8c8d-c063825aaee8 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:03.593588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ae4bba-17ae-48de-8c8d-c063825aaee8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:03.593893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ae4bba-17ae-48de-8c8d-c063825aaee8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:03.596404 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ae4bba-17ae-48de-8c8d-c063825aaee8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:03.608851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 542/1068] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:52:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:03.641934 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3b61d-f2c4-4532-8846-7d431df99efc None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:03.683774 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c6c30-a4af-4acc-9064-f68b5b9631a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:03.684683 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c6c30-a4af-4acc-9064-f68b5b9631a8 None admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:03.684990 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c6c30-a4af-4acc-9064-f68b5b9631a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:03.685264 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c6c30-a4af-4acc-9064-f68b5b9631a8 None admin] PATH_INFO: `/v3/projects/ca1d09f491d944f99425a641561b03c3/users/1899c317083d40f5b490cd9eb63fe521/roles/04c312d9dc3249fe8fbff78befef65f9` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:03.687545 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c6c30-a4af-4acc-9064-f68b5b9631a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1d09f491d944f99425a641561b03c3, user_id=1899c317083d40f5b490cd9eb63fe521, role_id=04c312d9dc3249fe8fbff78befef65f9)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:03.689280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c6c30-a4af-4acc-9064-f68b5b9631a8 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:03.705164 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 527/1069] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:52:03 2022] PUT /identity/v3/projects/ca1d09f491d944f99425a641561b03c3/users/1899c317083d40f5b490cd9eb63fe521/roles/04c312d9dc3249fe8fbff78befef65f9 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:52:03.931629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b542c33-453e-4634-a730-5205aa6aba29 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:03.931822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b542c33-453e-4634-a730-5205aa6aba29 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:03.932105 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b542c33-453e-4634-a730-5205aa6aba29 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:03.933501 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 543/1070] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:52:03.939260 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cff41c-c853-4f17-b507-d30c4899c23b None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:03.939549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cff41c-c853-4f17-b507-d30c4899c23b None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:03.939828 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cff41c-c853-4f17-b507-d30c4899c23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:03.957903 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-a3cff41c-c853-4f17-b507-d30c4899c23b None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:03.959128 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-a3cff41c-c853-4f17-b507-d30c4899c23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:04.020906 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:04.044096 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 528/1071] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:52:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:52:04.067202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b542c33-453e-4634-a730-5205aa6aba29 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:04.109604 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:04.181100 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b542c33-453e-4634-a730-5205aa6aba29 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:04.218375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83019c79-368e-4d80-a160-9ef340e00f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:04.219088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83019c79-368e-4d80-a160-9ef340e00f0d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:04.219360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83019c79-368e-4d80-a160-9ef340e00f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:04.219636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83019c79-368e-4d80-a160-9ef340e00f0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:04.231026 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83019c79-368e-4d80-a160-9ef340e00f0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:52:04.238736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-83019c79-368e-4d80-a160-9ef340e00f0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:52:04.238736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:52:04.238736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:52:04.238736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:52:04.238736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:52:04.238736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:52:04.238736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:52:04.238736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:52:04.238736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:52:04.238736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:52:04.238736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:52:04.238736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:52:04.238736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:52:04.238736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:52:04.238736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:52:04.238736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:52:04.239797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:52:04.239797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:52:04.239797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:52:04.239797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:52:04.239797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:52:04.239797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:52:04.239797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:52:04.239797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:52:04.239797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:52:04.239797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:52:04.239797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:52:04.239797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:52:04.239797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:52:04.239797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:52:04.239797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:52:04.239797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:52:04.240661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:52:04.240661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:52:04.240661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:52:04.240661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:52:04.240661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:52:04.240661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:52:04.240661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:52:04.240661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:52:04.240661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:52:04.240661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:52:04.240661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:52:04.240661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:52:04.240661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 544/1072] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:52:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:52:04.245902 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cff41c-c853-4f17-b507-d30c4899c23b None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:04.348396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cff41c-c853-4f17-b507-d30c4899c23b None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:04.382576 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21281635-94bb-499e-b74b-e2aae4fd6ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:04.383157 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21281635-94bb-499e-b74b-e2aae4fd6ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:04.383437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21281635-94bb-499e-b74b-e2aae4fd6ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:04.383710 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21281635-94bb-499e-b74b-e2aae4fd6ae1 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:04.384454 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21281635-94bb-499e-b74b-e2aae4fd6ae1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:04.384757 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21281635-94bb-499e-b74b-e2aae4fd6ae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:04.386127 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21281635-94bb-499e-b74b-e2aae4fd6ae1 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:04.395622 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 529/1073] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:52:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:04.401354 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83019c79-368e-4d80-a160-9ef340e00f0d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:04.499684 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:04.504943 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83019c79-368e-4d80-a160-9ef340e00f0d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:04.538557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51793a-6666-4cc3-8a2d-d6f8718e173e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:04.539417 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51793a-6666-4cc3-8a2d-d6f8718e173e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:04.539698 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51793a-6666-4cc3-8a2d-d6f8718e173e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:04.539973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51793a-6666-4cc3-8a2d-d6f8718e173e None admin] PATH_INFO: `/v3/users/123d56810d58464a9a806e4cb33d3650` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:04.554069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51793a-6666-4cc3-8a2d-d6f8718e173e None admin] RBAC: Authorizing `identity:get_user(user_id=123d56810d58464a9a806e4cb33d3650)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:04.555547 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51793a-6666-4cc3-8a2d-d6f8718e173e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:04.557271 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 545/1074] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:52:04 2022] GET /identity/v3/users/123d56810d58464a9a806e4cb33d3650 => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:52:04.563006 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21281635-94bb-499e-b74b-e2aae4fd6ae1 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:04.611574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:04.669000 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21281635-94bb-499e-b74b-e2aae4fd6ae1 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:04.703763 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839536a-6320-49d7-ab84-ec466de07849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:04.704463 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839536a-6320-49d7-ab84-ec466de07849 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:04.704746 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839536a-6320-49d7-ab84-ec466de07849 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:04.705033 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839536a-6320-49d7-ab84-ec466de07849 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:04.773155 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e839536a-6320-49d7-ab84-ec466de07849 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:04.774568 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e839536a-6320-49d7-ab84-ec466de07849 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:04.824216 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:04.881752 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 530/1075] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:52:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:52:04.887602 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51793a-6666-4cc3-8a2d-d6f8718e173e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:04.986941 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51793a-6666-4cc3-8a2d-d6f8718e173e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:05.022720 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96556ffa-9ea2-4c02-b6d1-89fa0b155c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:05.023781 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96556ffa-9ea2-4c02-b6d1-89fa0b155c49 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:05.024124 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96556ffa-9ea2-4c02-b6d1-89fa0b155c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:05.024399 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96556ffa-9ea2-4c02-b6d1-89fa0b155c49 None admin] PATH_INFO: `/v3/projects/cabe5626f5294da3a96fced92def22c6` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:05.025581 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96556ffa-9ea2-4c02-b6d1-89fa0b155c49 None admin] RBAC: Authorizing `identity:get_project(project_id=cabe5626f5294da3a96fced92def22c6)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:05.027220 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96556ffa-9ea2-4c02-b6d1-89fa0b155c49 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:05.029268 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 546/1076] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:52:04 2022] GET /identity/v3/projects/cabe5626f5294da3a96fced92def22c6 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:05.036380 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839536a-6320-49d7-ab84-ec466de07849 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:05.149906 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839536a-6320-49d7-ab84-ec466de07849 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:05.190466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa04f37-ccb8-4da4-a3c8-ac2c5ea5a772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:05.191182 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa04f37-ccb8-4da4-a3c8-ac2c5ea5a772 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:05.191461 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa04f37-ccb8-4da4-a3c8-ac2c5ea5a772 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:05.191734 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa04f37-ccb8-4da4-a3c8-ac2c5ea5a772 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:05.192568 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa04f37-ccb8-4da4-a3c8-ac2c5ea5a772 None admin] RBAC: Adding query filter params (role.id=4fa8f06085c741a5b3cf64e63cc0754f, scope.project.id=cabe5626f5294da3a96fced92def22c6, user.id=123d56810d58464a9a806e4cb33d3650) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:05.192919 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa04f37-ccb8-4da4-a3c8-ac2c5ea5a772 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:05.194534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa04f37-ccb8-4da4-a3c8-ac2c5ea5a772 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:05.200883 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 531/1077] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:52:05 2022] GET /identity/v3/role_assignments?user.id=123d56810d58464a9a806e4cb33d3650&scope.project.id=cabe5626f5294da3a96fced92def22c6&role.id=4fa8f06085c741a5b3cf64e63cc0754f => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:05.980914 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749793cd-9bea-490e-8827-19bcfd16ddea None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:05.981436 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749793cd-9bea-490e-8827-19bcfd16ddea None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:05.981528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749793cd-9bea-490e-8827-19bcfd16ddea None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:05.982883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 547/1078] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:52:05.988942 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480f39f-db6f-49ae-8f79-d3ff6dc2fea4 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:05.989401 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480f39f-db6f-49ae-8f79-d3ff6dc2fea4 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:05.989565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480f39f-db6f-49ae-8f79-d3ff6dc2fea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:06.008366 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-2480f39f-db6f-49ae-8f79-d3ff6dc2fea4 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:06.009568 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-2480f39f-db6f-49ae-8f79-d3ff6dc2fea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:06.011705 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:06.076980 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 532/1079] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:52:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:52:06.101197 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749793cd-9bea-490e-8827-19bcfd16ddea None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:06.126993 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:06.215610 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749793cd-9bea-490e-8827-19bcfd16ddea None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:06.252130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4878ed-d9e5-4a62-8059-b26d37742951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:06.252969 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4878ed-d9e5-4a62-8059-b26d37742951 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:06.253999 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4878ed-d9e5-4a62-8059-b26d37742951 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:06.253999 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4878ed-d9e5-4a62-8059-b26d37742951 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:06.264049 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4878ed-d9e5-4a62-8059-b26d37742951 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:52:06.272138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-5c4878ed-d9e5-4a62-8059-b26d37742951 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:52:06.272138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:52:06.272138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:52:06.272138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:52:06.272138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:52:06.272138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:52:06.272138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:52:06.272138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:52:06.272138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:52:06.272138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:52:06.272138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:52:06.272138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:52:06.272138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:52:06.272138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:52:06.272138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:52:06.272138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:52:06.273140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:52:06.273140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:52:06.273140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:52:06.273140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:52:06.273140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:52:06.273140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:52:06.273140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:52:06.273140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:52:06.273140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:52:06.273140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:52:06.273140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:52:06.273140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:52:06.273140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:52:06.273140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:52:06.273140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:52:06.273140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:52:06.274141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:52:06.274141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:52:06.274141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:52:06.274141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:52:06.274141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:52:06.274141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:52:06.274141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:52:06.274141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:52:06.274141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:52:06.274141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:52:06.274141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 17:52:06.274141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:52:06.274141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 548/1080] 10.222.0.76 () {56 vars in 1120 bytes} [Thu May 26 17:52:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:52:06.279618 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480f39f-db6f-49ae-8f79-d3ff6dc2fea4 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:06.397988 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480f39f-db6f-49ae-8f79-d3ff6dc2fea4 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:06.435346 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3f3fa-adde-4f0c-879f-47b5a7d5d28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:06.436024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3f3fa-adde-4f0c-879f-47b5a7d5d28c None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:06.436331 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3f3fa-adde-4f0c-879f-47b5a7d5d28c None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:06.436594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3f3fa-adde-4f0c-879f-47b5a7d5d28c None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:06.437349 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3f3fa-adde-4f0c-879f-47b5a7d5d28c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:06.437657 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3f3fa-adde-4f0c-879f-47b5a7d5d28c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:06.439760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3f3fa-adde-4f0c-879f-47b5a7d5d28c None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:06.445487 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:06.451854 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 533/1081] 10.222.0.76 () {56 vars in 1115 bytes} [Thu May 26 17:52:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:06.457532 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4878ed-d9e5-4a62-8059-b26d37742951 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:06.552561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:06.570503 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4878ed-d9e5-4a62-8059-b26d37742951 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:06.607190 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a38508-72ef-48c8-bc60-10135eb4bee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:06.607993 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a38508-72ef-48c8-bc60-10135eb4bee6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:06.608277 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a38508-72ef-48c8-bc60-10135eb4bee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:06.608552 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a38508-72ef-48c8-bc60-10135eb4bee6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:06.620241 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a38508-72ef-48c8-bc60-10135eb4bee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:06.621707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a38508-72ef-48c8-bc60-10135eb4bee6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:06.639883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-a9a38508-72ef-48c8-bc60-10135eb4bee6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:52:06.639883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:52:06.639883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:52:06.639883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:52:06.639883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:52:06.639883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:52:06.639883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:52:06.639883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:52:06.639883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:52:06.639883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:52:06.639883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:52:06.639883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:52:06.639883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:52:06.639883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:52:06.639883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:52:06.640855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:52:06.640855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:52:06.640855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:52:06.640855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:52:06.640855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:52:06.640855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:52:06.640855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:52:06.640855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:52:06.640855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:52:06.640855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:52:06.640855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:52:06.640855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:52:06.640855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:52:06.640855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:52:06.640855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:52:06.640855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:52:06.640855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:52:06.642517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:52:06.642517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:52:06.642517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:52:06.642517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:52:06.642517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 549/1082] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:52:06.646939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3f3fa-adde-4f0c-879f-47b5a7d5d28c None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:06.756595 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3f3fa-adde-4f0c-879f-47b5a7d5d28c None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:06.795917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b1953-004a-4679-81b2-6ca066fb708e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:06.796584 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b1953-004a-4679-81b2-6ca066fb708e None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:06.796941 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b1953-004a-4679-81b2-6ca066fb708e None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:06.797139 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b1953-004a-4679-81b2-6ca066fb708e None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:06.797865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b1953-004a-4679-81b2-6ca066fb708e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:06.798309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b1953-004a-4679-81b2-6ca066fb708e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:06.800726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b1953-004a-4679-81b2-6ca066fb708e None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:06.815400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 534/1083] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:06.820791 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a38508-72ef-48c8-bc60-10135eb4bee6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:06.930861 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a38508-72ef-48c8-bc60-10135eb4bee6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:06.966058 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18ab71-036b-46a8-ba39-230817d5f498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:06.966885 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18ab71-036b-46a8-ba39-230817d5f498 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:06.967167 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18ab71-036b-46a8-ba39-230817d5f498 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:06.967487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18ab71-036b-46a8-ba39-230817d5f498 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:06.970717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:06.978135 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18ab71-036b-46a8-ba39-230817d5f498 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:06.979620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18ab71-036b-46a8-ba39-230817d5f498 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:06.995234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-6d18ab71-036b-46a8-ba39-230817d5f498 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 17:52:06.995234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:52:06.995234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:52:06.995234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:52:06.995234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:52:06.995234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:52:06.995234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:52:06.995234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:52:06.995234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:52:06.995234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:52:06.995234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:52:06.995234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:52:06.995234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:52:06.995234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:52:06.995234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:52:06.995234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:52:06.997265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:52:06.997265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:52:06.997265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:52:06.997265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:52:06.997265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:52:06.997265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:52:06.997265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:52:06.997265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:52:06.997265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:52:06.997265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:52:06.997265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:52:06.997265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:52:06.997265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:52:06.997265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:52:06.997265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:52:06.997265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:52:06.997265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:52:06.997265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:52:06.999158 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:52:06.999158 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:52:06.999158 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:52:06.999158 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:52:06.999158 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:52:06.999158 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:52:06.999158 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:52:06.999158 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:52:06.999158 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 17:52:06.999158 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:52:06.999158 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 550/1084] 10.222.0.76 () {56 vars in 1093 bytes} [Thu May 26 17:52:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:52:07.003013 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b1953-004a-4679-81b2-6ca066fb708e None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:07.068233 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:07.124500 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b1953-004a-4679-81b2-6ca066fb708e None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:07.162815 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe866ac1-12a6-459f-b290-c306b99eb335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:07.163492 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe866ac1-12a6-459f-b290-c306b99eb335 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:07.163777 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe866ac1-12a6-459f-b290-c306b99eb335 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:07.164056 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe866ac1-12a6-459f-b290-c306b99eb335 None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:07.164891 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe866ac1-12a6-459f-b290-c306b99eb335 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:07.165201 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe866ac1-12a6-459f-b290-c306b99eb335 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:07.166653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe866ac1-12a6-459f-b290-c306b99eb335 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:07.184722 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 535/1085] 10.222.0.76 () {56 vars in 1133 bytes} [Thu May 26 17:52:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:07.191384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18ab71-036b-46a8-ba39-230817d5f498 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:07.292084 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18ab71-036b-46a8-ba39-230817d5f498 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:07.329936 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edeab5-90d6-497e-93d9-d3244c754dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:07.330618 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edeab5-90d6-497e-93d9-d3244c754dda None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:07.330900 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edeab5-90d6-497e-93d9-d3244c754dda None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:07.331179 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edeab5-90d6-497e-93d9-d3244c754dda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:07.393921 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:07.401199 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edeab5-90d6-497e-93d9-d3244c754dda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:07.402777 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edeab5-90d6-497e-93d9-d3244c754dda None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:07.492727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980c5d9-667e-478e-9342-e66d53eca7f6 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:07.493193 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980c5d9-667e-478e-9342-e66d53eca7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:07.493661 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980c5d9-667e-478e-9342-e66d53eca7f6 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:07.496036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 536/1086] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:52:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:52:07.503564 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34389d2f-5e95-4994-a8be-2c3a4320aa93 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:07.503852 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34389d2f-5e95-4994-a8be-2c3a4320aa93 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:07.504131 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34389d2f-5e95-4994-a8be-2c3a4320aa93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:07.506546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 551/1087] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:52:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:52:07.512395 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edeab5-90d6-497e-93d9-d3244c754dda None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:07.525631 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-34389d2f-5e95-4994-a8be-2c3a4320aa93 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:07.526898 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-34389d2f-5e95-4994-a8be-2c3a4320aa93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:07.609110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 537/1088] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:52:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:52:07.616741 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:07.623444 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edeab5-90d6-497e-93d9-d3244c754dda None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:07.632175 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34389d2f-5e95-4994-a8be-2c3a4320aa93 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:07.659456 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edff8b5-f18f-4c32-a1f1-05805c431e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:07.660243 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edff8b5-f18f-4c32-a1f1-05805c431e56 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:07.660524 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edff8b5-f18f-4c32-a1f1-05805c431e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:07.661045 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edff8b5-f18f-4c32-a1f1-05805c431e56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:07.670965 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edff8b5-f18f-4c32-a1f1-05805c431e56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:07.672548 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edff8b5-f18f-4c32-a1f1-05805c431e56 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:07.682793 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:07.691114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-8edff8b5-f18f-4c32-a1f1-05805c431e56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:52:07.691114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:52:07.691114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:52:07.691114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:52:07.691114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:52:07.691114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:52:07.691114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:52:07.691114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:52:07.691114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:52:07.691114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:52:07.691114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:52:07.691114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:52:07.691114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:52:07.691114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:52:07.691114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:52:07.692202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:52:07.692202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:52:07.692202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:52:07.692202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:52:07.692202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:52:07.692202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:52:07.692202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:52:07.692202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:52:07.692202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:52:07.692202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:52:07.692202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:52:07.692202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:52:07.692202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:52:07.692202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:52:07.692202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:52:07.692202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:52:07.692202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:52:07.693250 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:52:07.693250 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:52:07.693250 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:52:07.693250 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:52:07.693250 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 552/1089] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:52:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:52:07.697883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edff8b5-f18f-4c32-a1f1-05805c431e56 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:07.737886 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34389d2f-5e95-4994-a8be-2c3a4320aa93 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:07.772049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5394c-c406-472f-8b7f-462d52378b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:07.772961 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5394c-c406-472f-8b7f-462d52378b3c None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:07.773247 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5394c-c406-472f-8b7f-462d52378b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:07.773522 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5394c-c406-472f-8b7f-462d52378b3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:07.788805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5394c-c406-472f-8b7f-462d52378b3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:52:07.798341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-15a5394c-c406-472f-8b7f-462d52378b3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:52:07.798341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:52:07.798341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:52:07.798341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:52:07.798341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:52:07.798341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:52:07.798341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:52:07.798341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:52:07.798341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:52:07.798341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:52:07.798341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:52:07.798341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:52:07.798341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:52:07.798341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:52:07.798341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:52:07.798341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:52:07.799499 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:52:07.799499 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:52:07.799499 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:52:07.799499 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:52:07.799499 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:52:07.799499 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:52:07.799499 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:52:07.799499 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:52:07.799499 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:52:07.799499 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:52:07.799499 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:52:07.799499 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:52:07.799499 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:52:07.799499 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:52:07.799499 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:52:07.799499 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:52:07.800575 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:52:07.800575 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:52:07.800575 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:52:07.800575 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:52:07.800575 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:52:07.800575 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:52:07.800575 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:52:07.800575 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:52:07.800575 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:52:07.800575 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:52:07.800575 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:52:07.800575 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:52:07.800575 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 538/1090] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:52:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:52:07.803874 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edff8b5-f18f-4c32-a1f1-05805c431e56 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:07.806015 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5394c-c406-472f-8b7f-462d52378b3c None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:07.839024 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83687f59-b84c-4fb3-b947-189ba9ecb54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:07.839686 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83687f59-b84c-4fb3-b947-189ba9ecb54a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:07.839967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83687f59-b84c-4fb3-b947-189ba9ecb54a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:07.840240 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83687f59-b84c-4fb3-b947-189ba9ecb54a None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:07.840947 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83687f59-b84c-4fb3-b947-189ba9ecb54a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:07.841267 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83687f59-b84c-4fb3-b947-189ba9ecb54a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:07.842670 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83687f59-b84c-4fb3-b947-189ba9ecb54a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:07.854825 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 553/1091] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:52:07.861542 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83687f59-b84c-4fb3-b947-189ba9ecb54a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:07.873168 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:07.921418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5394c-c406-472f-8b7f-462d52378b3c None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:07.931431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:07.959351 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c5ed4-c988-4743-a182-a28b63864458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:07.960040 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c5ed4-c988-4743-a182-a28b63864458 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:07.960332 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c5ed4-c988-4743-a182-a28b63864458 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:07.960647 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c5ed4-c988-4743-a182-a28b63864458 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:07.961417 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c5ed4-c988-4743-a182-a28b63864458 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:07.961728 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c5ed4-c988-4743-a182-a28b63864458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:07.963198 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c5ed4-c988-4743-a182-a28b63864458 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:07.973578 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 539/1092] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:52:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:52:07.975507 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83687f59-b84c-4fb3-b947-189ba9ecb54a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:07.979621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c5ed4-c988-4743-a182-a28b63864458 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:08.015968 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc9492-5fe3-474e-931e-e29cd3eb4c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:08.017392 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc9492-5fe3-474e-931e-e29cd3eb4c5e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:08.017746 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc9492-5fe3-474e-931e-e29cd3eb4c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:08.017980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc9492-5fe3-474e-931e-e29cd3eb4c5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:08.036106 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc9492-5fe3-474e-931e-e29cd3eb4c5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:08.037583 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc9492-5fe3-474e-931e-e29cd3eb4c5e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:08.060335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-5fdc9492-5fe3-474e-931e-e29cd3eb4c5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:52:08.060335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:52:08.060335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:52:08.060335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:52:08.060335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:52:08.060335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:52:08.060335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:52:08.060335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:52:08.060335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:52:08.060335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:52:08.060335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:52:08.060335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:52:08.060335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:52:08.060335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:52:08.060335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:52:08.061334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:52:08.061334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:52:08.061334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:52:08.061334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:52:08.061334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:52:08.061334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:52:08.061334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:52:08.061334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:52:08.061334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:52:08.061334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:52:08.061334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:52:08.061334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:52:08.061334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:52:08.061334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:52:08.061334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:52:08.061334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:52:08.061334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:52:08.062286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:52:08.062286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:52:08.062286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:52:08.062286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:52:08.062286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:52:08.062286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:52:08.062286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:52:08.062286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:52:08.062286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 554/1093] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:52:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:52:08.069918 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc9492-5fe3-474e-931e-e29cd3eb4c5e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:08.104257 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c5ed4-c988-4743-a182-a28b63864458 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:08.139917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af659e33-5964-4342-8439-7bbdb8deffb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:08.141037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af659e33-5964-4342-8439-7bbdb8deffb9 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:08.141498 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af659e33-5964-4342-8439-7bbdb8deffb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:08.142681 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af659e33-5964-4342-8439-7bbdb8deffb9 None admin] PATH_INFO: `/v3/users/123d56810d58464a9a806e4cb33d3650` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:08.144102 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af659e33-5964-4342-8439-7bbdb8deffb9 None admin] RBAC: Authorizing `identity:get_user(user_id=123d56810d58464a9a806e4cb33d3650)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:08.145406 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:08.145556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af659e33-5964-4342-8439-7bbdb8deffb9 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:08.147618 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 540/1094] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:52:07 2022] GET /identity/v3/users/123d56810d58464a9a806e4cb33d3650 => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:52:08.153514 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af659e33-5964-4342-8439-7bbdb8deffb9 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:08.183999 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc9492-5fe3-474e-931e-e29cd3eb4c5e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:08.196280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:08.222676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355f533-be58-4a2b-9af8-e37c7d53499d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:08.223343 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355f533-be58-4a2b-9af8-e37c7d53499d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:08.223654 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355f533-be58-4a2b-9af8-e37c7d53499d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:08.223904 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355f533-be58-4a2b-9af8-e37c7d53499d None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:08.224702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355f533-be58-4a2b-9af8-e37c7d53499d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:08.225023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355f533-be58-4a2b-9af8-e37c7d53499d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:08.226493 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355f533-be58-4a2b-9af8-e37c7d53499d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:08.238697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 555/1095] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:52:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:52:08.244645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355f533-be58-4a2b-9af8-e37c7d53499d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:08.265705 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af659e33-5964-4342-8439-7bbdb8deffb9 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:08.304846 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb18f4f-91c7-476b-88c8-eabee30b43db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:08.305715 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb18f4f-91c7-476b-88c8-eabee30b43db None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:08.306002 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb18f4f-91c7-476b-88c8-eabee30b43db None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:08.306319 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb18f4f-91c7-476b-88c8-eabee30b43db None admin] PATH_INFO: `/v3/projects/cabe5626f5294da3a96fced92def22c6` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:08.307510 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb18f4f-91c7-476b-88c8-eabee30b43db None admin] RBAC: Authorizing `identity:get_project(project_id=cabe5626f5294da3a96fced92def22c6)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:08.308952 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb18f4f-91c7-476b-88c8-eabee30b43db None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:08.310701 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 541/1096] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:52:08 2022] GET /identity/v3/projects/cabe5626f5294da3a96fced92def22c6 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:08.317914 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb18f4f-91c7-476b-88c8-eabee30b43db None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:08.352153 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355f533-be58-4a2b-9af8-e37c7d53499d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:08.380676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:08.389109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8dd617-a487-484b-89e2-33fb91ac590d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:08.389778 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8dd617-a487-484b-89e2-33fb91ac590d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:08.390105 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8dd617-a487-484b-89e2-33fb91ac590d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:08.390372 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8dd617-a487-484b-89e2-33fb91ac590d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:08.391197 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8dd617-a487-484b-89e2-33fb91ac590d None admin] RBAC: Adding query filter params (role.id=04c312d9dc3249fe8fbff78befef65f9, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=1899c317083d40f5b490cd9eb63fe521) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:08.391508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8dd617-a487-484b-89e2-33fb91ac590d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:08.393688 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8dd617-a487-484b-89e2-33fb91ac590d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:08.400152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 556/1097] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:52:08 2022] GET /identity/v3/role_assignments?user.id=1899c317083d40f5b490cd9eb63fe521&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=04c312d9dc3249fe8fbff78befef65f9 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:52:08.424123 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb18f4f-91c7-476b-88c8-eabee30b43db None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:08.433071 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:08.463815 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77087f8c-8ad6-4e34-bf42-19a29e086229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:08.464733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77087f8c-8ad6-4e34-bf42-19a29e086229 None admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:08.465019 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77087f8c-8ad6-4e34-bf42-19a29e086229 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:08.465303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77087f8c-8ad6-4e34-bf42-19a29e086229 None admin] PATH_INFO: `/v3/projects/cabe5626f5294da3a96fced92def22c6/users/123d56810d58464a9a806e4cb33d3650/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:08.467485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77087f8c-8ad6-4e34-bf42-19a29e086229 None admin] RBAC: Authorizing `identity:create_grant(project_id=cabe5626f5294da3a96fced92def22c6, user_id=123d56810d58464a9a806e4cb33d3650, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:08.468976 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77087f8c-8ad6-4e34-bf42-19a29e086229 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:08.484306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 542/1098] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:52:08 2022] PUT /identity/v3/projects/cabe5626f5294da3a96fced92def22c6/users/123d56810d58464a9a806e4cb33d3650/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 17:52:10.852799 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f8ef7-42fe-4576-b285-323975eca6a8 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:10.853667 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f8ef7-42fe-4576-b285-323975eca6a8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:10.853667 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f8ef7-42fe-4576-b285-323975eca6a8 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:10.854836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 557/1099] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:52:10.861147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e8e97-06fd-4f28-8cf4-d181352a08e5 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:10.861722 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e8e97-06fd-4f28-8cf4-d181352a08e5 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:10.861872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e8e97-06fd-4f28-8cf4-d181352a08e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:10.879518 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-5b0e8e97-06fd-4f28-8cf4-d181352a08e5 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:10.880722 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-5b0e8e97-06fd-4f28-8cf4-d181352a08e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:10.958052 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 543/1100] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:52:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:52:10.983038 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f8ef7-42fe-4576-b285-323975eca6a8 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:11.089369 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f8ef7-42fe-4576-b285-323975eca6a8 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:11.123424 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240e49d-7ab9-42d4-831e-15bdc4662d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:11.124136 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240e49d-7ab9-42d4-831e-15bdc4662d9f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:11.124420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240e49d-7ab9-42d4-831e-15bdc4662d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:11.124696 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240e49d-7ab9-42d4-831e-15bdc4662d9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:11.135913 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240e49d-7ab9-42d4-831e-15bdc4662d9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:52:11.144378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-2240e49d-7ab9-42d4-831e-15bdc4662d9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:52:11.144378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:52:11.144378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:52:11.144378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:52:11.144378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:52:11.144378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:52:11.144378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:52:11.144378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:52:11.144378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:52:11.144378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:52:11.144378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:52:11.144378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:52:11.144378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:52:11.144378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:52:11.144378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:52:11.144378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:52:11.145452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:52:11.145452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:52:11.145452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:52:11.145452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:52:11.145452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:52:11.145452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:52:11.145452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:52:11.145452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:52:11.145452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:52:11.145452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:52:11.145452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:52:11.145452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:52:11.145452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:52:11.145452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:52:11.145452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:52:11.145452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:52:11.146600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:52:11.146600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:52:11.146600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:52:11.146600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:52:11.146600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:52:11.146600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:52:11.146600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:52:11.146600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:52:11.146600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:52:11.146600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:52:11.146600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:52:11.146600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:52:11.146600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 558/1101] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:52:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:52:11.151730 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e8e97-06fd-4f28-8cf4-d181352a08e5 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:11.249410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:11.256560 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e8e97-06fd-4f28-8cf4-d181352a08e5 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:11.298827 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837cd5e-16a1-482d-93f9-aaef2781464c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:11.299606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837cd5e-16a1-482d-93f9-aaef2781464c None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:11.299893 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837cd5e-16a1-482d-93f9-aaef2781464c None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:11.300173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837cd5e-16a1-482d-93f9-aaef2781464c None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:11.300925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837cd5e-16a1-482d-93f9-aaef2781464c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:11.301237 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837cd5e-16a1-482d-93f9-aaef2781464c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:11.302662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837cd5e-16a1-482d-93f9-aaef2781464c None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:11.311395 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 544/1102] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:52:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:11.318140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240e49d-7ab9-42d4-831e-15bdc4662d9f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:11.372090 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:11.426244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240e49d-7ab9-42d4-831e-15bdc4662d9f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:11.461056 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aed879-8b4a-42fe-8920-6f3ca4509444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:11.461917 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aed879-8b4a-42fe-8920-6f3ca4509444 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:11.462221 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aed879-8b4a-42fe-8920-6f3ca4509444 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:11.462504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aed879-8b4a-42fe-8920-6f3ca4509444 None admin] PATH_INFO: `/v3/users/123d56810d58464a9a806e4cb33d3650` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:11.463702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aed879-8b4a-42fe-8920-6f3ca4509444 None admin] RBAC: Authorizing `identity:get_user(user_id=123d56810d58464a9a806e4cb33d3650)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:11.465160 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aed879-8b4a-42fe-8920-6f3ca4509444 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:11.466851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 559/1103] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:52:11 2022] GET /identity/v3/users/123d56810d58464a9a806e4cb33d3650 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:11.472772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837cd5e-16a1-482d-93f9-aaef2781464c None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:11.581048 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837cd5e-16a1-482d-93f9-aaef2781464c None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:11.614809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6173b0-6388-460d-927c-d25991842a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:11.615454 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6173b0-6388-460d-927c-d25991842a07 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:11.615737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6173b0-6388-460d-927c-d25991842a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:11.616011 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6173b0-6388-460d-927c-d25991842a07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:11.642987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:11.683276 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6173b0-6388-460d-927c-d25991842a07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:11.684651 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6173b0-6388-460d-927c-d25991842a07 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:11.785917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 545/1104] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:52:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:52:11.792018 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aed879-8b4a-42fe-8920-6f3ca4509444 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:11.895817 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aed879-8b4a-42fe-8920-6f3ca4509444 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:11.916663 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:11.932204 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb186229-7d02-4820-a320-e5a710677279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:11.933063 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb186229-7d02-4820-a320-e5a710677279 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:11.933398 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb186229-7d02-4820-a320-e5a710677279 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:11.933618 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb186229-7d02-4820-a320-e5a710677279 None admin] PATH_INFO: `/v3/projects/cabe5626f5294da3a96fced92def22c6` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:11.934818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb186229-7d02-4820-a320-e5a710677279 None admin] RBAC: Authorizing `identity:get_project(project_id=cabe5626f5294da3a96fced92def22c6)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:11.936256 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb186229-7d02-4820-a320-e5a710677279 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:11.938008 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 560/1105] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:52:11 2022] GET /identity/v3/projects/cabe5626f5294da3a96fced92def22c6 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:11.947939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6173b0-6388-460d-927c-d25991842a07 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:12.016140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:12.061395 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6173b0-6388-460d-927c-d25991842a07 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:12.095389 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dac9ea-8345-49b8-88a5-0a8c110c588b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:12.096498 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dac9ea-8345-49b8-88a5-0a8c110c588b None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:12.096778 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dac9ea-8345-49b8-88a5-0a8c110c588b None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:12.097050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dac9ea-8345-49b8-88a5-0a8c110c588b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:12.097865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dac9ea-8345-49b8-88a5-0a8c110c588b None admin] RBAC: Adding query filter params (role.id=4fa8f06085c741a5b3cf64e63cc0754f, scope.project.id=cabe5626f5294da3a96fced92def22c6, user.id=123d56810d58464a9a806e4cb33d3650) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:12.098173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dac9ea-8345-49b8-88a5-0a8c110c588b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:12.099621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dac9ea-8345-49b8-88a5-0a8c110c588b None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:12.105296 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 546/1106] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:52:11 2022] GET /identity/v3/role_assignments?user.id=123d56810d58464a9a806e4cb33d3650&scope.project.id=cabe5626f5294da3a96fced92def22c6&role.id=4fa8f06085c741a5b3cf64e63cc0754f => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:14.418821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d6ff2-0d92-424d-a35d-3016febfbd9d None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:14.419330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d6ff2-0d92-424d-a35d-3016febfbd9d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:14.419330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d6ff2-0d92-424d-a35d-3016febfbd9d None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:14.420664 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 561/1107] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:52:14.427390 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886e4f6-0c1b-417d-b574-1cfe9b2b57d4 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:14.427868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886e4f6-0c1b-417d-b574-1cfe9b2b57d4 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:14.427958 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886e4f6-0c1b-417d-b574-1cfe9b2b57d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:14.445942 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-8886e4f6-0c1b-417d-b574-1cfe9b2b57d4 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:14.447167 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-8886e4f6-0c1b-417d-b574-1cfe9b2b57d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:14.519222 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 547/1108] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:52:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:52:14.544841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d6ff2-0d92-424d-a35d-3016febfbd9d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:14.653870 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d6ff2-0d92-424d-a35d-3016febfbd9d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:14.690445 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38d152-c7f0-43b5-b580-8aea5d8122eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:14.691159 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38d152-c7f0-43b5-b580-8aea5d8122eb None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:14.691440 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38d152-c7f0-43b5-b580-8aea5d8122eb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:14.691715 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38d152-c7f0-43b5-b580-8aea5d8122eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:14.692893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38d152-c7f0-43b5-b580-8aea5d8122eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:14.694353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38d152-c7f0-43b5-b580-8aea5d8122eb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:14.695971 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 562/1109] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:52:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:14.702527 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886e4f6-0c1b-417d-b574-1cfe9b2b57d4 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:14.757638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:14.808427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886e4f6-0c1b-417d-b574-1cfe9b2b57d4 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:14.845506 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee159e24-42c5-418e-994a-6df54608972f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:14.846553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee159e24-42c5-418e-994a-6df54608972f None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:14.846836 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee159e24-42c5-418e-994a-6df54608972f None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:14.847109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee159e24-42c5-418e-994a-6df54608972f None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:14.847827 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee159e24-42c5-418e-994a-6df54608972f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:14.849275 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee159e24-42c5-418e-994a-6df54608972f None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:14.872276 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 548/1110] 10.222.0.76 () {60 vars in 1132 bytes} [Thu May 26 17:52:14 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:52:17.242597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c153757-43d1-4f28-b852-d025cef523cd None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:17.243153 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c153757-43d1-4f28-b852-d025cef523cd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:17.243259 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c153757-43d1-4f28-b852-d025cef523cd None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:17.244651 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 563/1111] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:52:17.251355 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ffae3-9212-4260-ada3-807a75626345 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:17.251872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ffae3-9212-4260-ada3-807a75626345 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:17.252087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ffae3-9212-4260-ada3-807a75626345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:17.271904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-f91ffae3-9212-4260-ada3-807a75626345 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:17.273172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f91ffae3-9212-4260-ada3-807a75626345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:17.348154 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 549/1112] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:52:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:52:17.372369 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c153757-43d1-4f28-b852-d025cef523cd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:17.435434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:17.492614 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c153757-43d1-4f28-b852-d025cef523cd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:17.548824 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942c119-ef69-427e-8d56-ee6a6b37a44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:17.550273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942c119-ef69-427e-8d56-ee6a6b37a44e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:17.550760 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942c119-ef69-427e-8d56-ee6a6b37a44e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:17.551248 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942c119-ef69-427e-8d56-ee6a6b37a44e None admin] PATH_INFO: `/v3/roles/e731efeb1d3c407fb58cf64a02d95796` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:17.553772 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7942c119-ef69-427e-8d56-ee6a6b37a44e None admin] RBAC: Authorizing `identity:get_role(role_id=e731efeb1d3c407fb58cf64a02d95796)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:17.556218 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7942c119-ef69-427e-8d56-ee6a6b37a44e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:17.558713 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 564/1113] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:52:17 2022] GET /identity/v3/roles/e731efeb1d3c407fb58cf64a02d95796 => generated 223 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:17.564712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ffae3-9212-4260-ada3-807a75626345 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:17.607323 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:17.675453 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ffae3-9212-4260-ada3-807a75626345 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:17.717143 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc705f81-2cfc-402e-98ba-e87511f4dda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:17.718012 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc705f81-2cfc-402e-98ba-e87511f4dda0 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:17.718389 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc705f81-2cfc-402e-98ba-e87511f4dda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:17.718720 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc705f81-2cfc-402e-98ba-e87511f4dda0 None admin] PATH_INFO: `/v3/users/7dd07d2e54474c0c84b45fb6227e7e33` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:17.735165 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc705f81-2cfc-402e-98ba-e87511f4dda0 None admin] RBAC: Authorizing `identity:get_user(user_id=7dd07d2e54474c0c84b45fb6227e7e33)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:17.737522 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc705f81-2cfc-402e-98ba-e87511f4dda0 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:17.739406 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 550/1114] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:52:17 2022] GET /identity/v3/users/7dd07d2e54474c0c84b45fb6227e7e33 => generated 286 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:17.745382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942c119-ef69-427e-8d56-ee6a6b37a44e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:17.861994 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942c119-ef69-427e-8d56-ee6a6b37a44e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:17.889241 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:17.902404 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86129c-2fe3-4f48-8810-5cc492ef9398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:17.903066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86129c-2fe3-4f48-8810-5cc492ef9398 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:17.903351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86129c-2fe3-4f48-8810-5cc492ef9398 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:17.903629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86129c-2fe3-4f48-8810-5cc492ef9398 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:17.975633 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86129c-2fe3-4f48-8810-5cc492ef9398 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:17.977186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86129c-2fe3-4f48-8810-5cc492ef9398 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:18.088493 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 565/1115] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:52:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:52:18.095974 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc705f81-2cfc-402e-98ba-e87511f4dda0 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:18.240104 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:18.250728 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc705f81-2cfc-402e-98ba-e87511f4dda0 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:18.299024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3bf46-331c-4c4f-b1b1-a493450b1689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:18.299965 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3bf46-331c-4c4f-b1b1-a493450b1689 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:18.300290 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3bf46-331c-4c4f-b1b1-a493450b1689 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:18.300607 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3bf46-331c-4c4f-b1b1-a493450b1689 None admin] PATH_INFO: `/v3/projects/cabe5626f5294da3a96fced92def22c6` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:18.301911 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a3bf46-331c-4c4f-b1b1-a493450b1689 None admin] RBAC: Authorizing `identity:get_project(project_id=cabe5626f5294da3a96fced92def22c6)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:18.303536 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a3bf46-331c-4c4f-b1b1-a493450b1689 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:18.305321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 551/1116] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:52:18 2022] GET /identity/v3/projects/cabe5626f5294da3a96fced92def22c6 => generated 311 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:52:18.311529 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86129c-2fe3-4f48-8810-5cc492ef9398 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:18.338629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:18.419321 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86129c-2fe3-4f48-8810-5cc492ef9398 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:18.453118 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c25f9d-7510-4bbe-8fc0-363c9d25b31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:18.453776 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c25f9d-7510-4bbe-8fc0-363c9d25b31a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:18.454671 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c25f9d-7510-4bbe-8fc0-363c9d25b31a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:18.455147 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c25f9d-7510-4bbe-8fc0-363c9d25b31a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:18.455980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c25f9d-7510-4bbe-8fc0-363c9d25b31a None admin] RBAC: Adding query filter params (role.id=e731efeb1d3c407fb58cf64a02d95796, scope.project.id=cabe5626f5294da3a96fced92def22c6, user.id=7dd07d2e54474c0c84b45fb6227e7e33) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:18.456293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c25f9d-7510-4bbe-8fc0-363c9d25b31a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:18.457747 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c25f9d-7510-4bbe-8fc0-363c9d25b31a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:18.463629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 566/1117] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:52:18 2022] GET /identity/v3/role_assignments?user.id=7dd07d2e54474c0c84b45fb6227e7e33&scope.project.id=cabe5626f5294da3a96fced92def22c6&role.id=e731efeb1d3c407fb58cf64a02d95796 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:20.788708 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674150ad-8a16-425b-b58f-fd86103d7cbf None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:20.789339 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674150ad-8a16-425b-b58f-fd86103d7cbf None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:20.789339 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674150ad-8a16-425b-b58f-fd86103d7cbf None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:20.790684 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 552/1118] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:52:20.796941 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76acb6-3bf3-4b14-aa67-c5bc6c8122a9 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:20.797501 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76acb6-3bf3-4b14-aa67-c5bc6c8122a9 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:20.797501 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76acb6-3bf3-4b14-aa67-c5bc6c8122a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:20.816002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-af76acb6-3bf3-4b14-aa67-c5bc6c8122a9 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:20.817213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-af76acb6-3bf3-4b14-aa67-c5bc6c8122a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:20.890622 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 567/1119] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:52:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:52:20.914936 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674150ad-8a16-425b-b58f-fd86103d7cbf None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:21.035388 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674150ad-8a16-425b-b58f-fd86103d7cbf None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:21.074079 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9d245-2210-41ac-9e6f-4fe25944afe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:21.074869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9d245-2210-41ac-9e6f-4fe25944afe9 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:21.075155 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9d245-2210-41ac-9e6f-4fe25944afe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:21.075433 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9d245-2210-41ac-9e6f-4fe25944afe9 None admin] PATH_INFO: `/v3/roles/e731efeb1d3c407fb58cf64a02d95796` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:21.076950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9d245-2210-41ac-9e6f-4fe25944afe9 None admin] RBAC: Authorizing `identity:get_role(role_id=e731efeb1d3c407fb58cf64a02d95796)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:21.078375 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9d245-2210-41ac-9e6f-4fe25944afe9 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:21.080327 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 553/1120] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:52:20 2022] GET /identity/v3/roles/e731efeb1d3c407fb58cf64a02d95796 => generated 223 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:52:21.086556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76acb6-3bf3-4b14-aa67-c5bc6c8122a9 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:21.106345 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:21.195467 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76acb6-3bf3-4b14-aa67-c5bc6c8122a9 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:21.231492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee90bfd-8d9c-4055-b33e-1b504d58c1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:21.232428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee90bfd-8d9c-4055-b33e-1b504d58c1ea None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:21.232747 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee90bfd-8d9c-4055-b33e-1b504d58c1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:21.233060 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee90bfd-8d9c-4055-b33e-1b504d58c1ea None admin] PATH_INFO: `/v3/users/7dd07d2e54474c0c84b45fb6227e7e33` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:21.234368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee90bfd-8d9c-4055-b33e-1b504d58c1ea None admin] RBAC: Authorizing `identity:get_user(user_id=7dd07d2e54474c0c84b45fb6227e7e33)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:21.235921 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee90bfd-8d9c-4055-b33e-1b504d58c1ea None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:21.237823 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 568/1121] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:52:21 2022] GET /identity/v3/users/7dd07d2e54474c0c84b45fb6227e7e33 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:52:21.243796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9d245-2210-41ac-9e6f-4fe25944afe9 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:21.278324 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:21.353925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9d245-2210-41ac-9e6f-4fe25944afe9 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:21.389333 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1c7c2-d195-4222-880d-cb048aa67e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:21.390229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1c7c2-d195-4222-880d-cb048aa67e27 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:21.390515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1c7c2-d195-4222-880d-cb048aa67e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:21.390791 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1c7c2-d195-4222-880d-cb048aa67e27 None admin] PATH_INFO: `/v3/projects/cabe5626f5294da3a96fced92def22c6` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:21.393011 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1c7c2-d195-4222-880d-cb048aa67e27 None admin] RBAC: Authorizing `identity:get_project(project_id=cabe5626f5294da3a96fced92def22c6)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:21.394493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1c7c2-d195-4222-880d-cb048aa67e27 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:21.396235 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 554/1122] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:52:21 2022] GET /identity/v3/projects/cabe5626f5294da3a96fced92def22c6 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:21.402391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee90bfd-8d9c-4055-b33e-1b504d58c1ea None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:21.493778 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:21.512544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee90bfd-8d9c-4055-b33e-1b504d58c1ea None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:21.549059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3174c-7c1a-4990-9fb9-f1e6403f4e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:21.549952 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3174c-7c1a-4990-9fb9-f1e6403f4e46 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:21.550258 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3174c-7c1a-4990-9fb9-f1e6403f4e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:21.550538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3174c-7c1a-4990-9fb9-f1e6403f4e46 None admin] PATH_INFO: `/v3/projects/cabe5626f5294da3a96fced92def22c6/users/7dd07d2e54474c0c84b45fb6227e7e33/roles/e731efeb1d3c407fb58cf64a02d95796` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:21.552736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3174c-7c1a-4990-9fb9-f1e6403f4e46 None admin] RBAC: Authorizing `identity:create_grant(project_id=cabe5626f5294da3a96fced92def22c6, user_id=7dd07d2e54474c0c84b45fb6227e7e33, role_id=e731efeb1d3c407fb58cf64a02d95796)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:21.554207 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3174c-7c1a-4990-9fb9-f1e6403f4e46 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:21.568764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 569/1123] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:52:21 2022] PUT /identity/v3/projects/cabe5626f5294da3a96fced92def22c6/users/7dd07d2e54474c0c84b45fb6227e7e33/roles/e731efeb1d3c407fb58cf64a02d95796 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 17:52:23.900175 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86625299-4952-423a-af9b-5db8eee1dfb1 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:23.900820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86625299-4952-423a-af9b-5db8eee1dfb1 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:23.900820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86625299-4952-423a-af9b-5db8eee1dfb1 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:23.902078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 555/1124] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:52:23.907670 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d264f-7076-4d88-95ee-1e19d111d4d1 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:23.908163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d264f-7076-4d88-95ee-1e19d111d4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:23.908265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d264f-7076-4d88-95ee-1e19d111d4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:23.925770 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-874d264f-7076-4d88-95ee-1e19d111d4d1 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:23.927049 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-874d264f-7076-4d88-95ee-1e19d111d4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:23.995137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 570/1125] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:52:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:52:24.020487 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86625299-4952-423a-af9b-5db8eee1dfb1 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:24.134997 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:24.143130 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86625299-4952-423a-af9b-5db8eee1dfb1 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:24.203166 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e4be4-b324-421b-bf08-ce1bea3315e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:24.204753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e4be4-b324-421b-bf08-ce1bea3315e1 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:24.205196 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e4be4-b324-421b-bf08-ce1bea3315e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:24.205482 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e4be4-b324-421b-bf08-ce1bea3315e1 None admin] PATH_INFO: `/v3/roles/e731efeb1d3c407fb58cf64a02d95796` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:24.207093 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e4be4-b324-421b-bf08-ce1bea3315e1 None admin] RBAC: Authorizing `identity:get_role(role_id=e731efeb1d3c407fb58cf64a02d95796)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:24.208500 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e4be4-b324-421b-bf08-ce1bea3315e1 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:24.209898 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 556/1126] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:52:24 2022] GET /identity/v3/roles/e731efeb1d3c407fb58cf64a02d95796 => generated 223 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:24.215897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d264f-7076-4d88-95ee-1e19d111d4d1 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:24.300847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:24.328663 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d264f-7076-4d88-95ee-1e19d111d4d1 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:24.365283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c834e-959b-42e3-90a1-16f7aadd6284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:24.366172 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c834e-959b-42e3-90a1-16f7aadd6284 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:24.366485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c834e-959b-42e3-90a1-16f7aadd6284 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:24.366762 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c834e-959b-42e3-90a1-16f7aadd6284 None admin] PATH_INFO: `/v3/users/7dd07d2e54474c0c84b45fb6227e7e33` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:24.367981 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c834e-959b-42e3-90a1-16f7aadd6284 None admin] RBAC: Authorizing `identity:get_user(user_id=7dd07d2e54474c0c84b45fb6227e7e33)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:24.369471 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c834e-959b-42e3-90a1-16f7aadd6284 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:24.371167 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 571/1127] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:52:24 2022] GET /identity/v3/users/7dd07d2e54474c0c84b45fb6227e7e33 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:24.378221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e4be4-b324-421b-bf08-ce1bea3315e1 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:24.484266 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e4be4-b324-421b-bf08-ce1bea3315e1 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:24.521133 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971c8de-7b65-427a-9290-e8d99dc9fa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:24.521792 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971c8de-7b65-427a-9290-e8d99dc9fa0a None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:24.522074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971c8de-7b65-427a-9290-e8d99dc9fa0a None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:24.522375 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971c8de-7b65-427a-9290-e8d99dc9fa0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:24.559439 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:24.597529 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971c8de-7b65-427a-9290-e8d99dc9fa0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:24.599016 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971c8de-7b65-427a-9290-e8d99dc9fa0a None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:24.704056 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 557/1128] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:52:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 17:52:24.710700 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c834e-959b-42e3-90a1-16f7aadd6284 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:24.817215 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c834e-959b-42e3-90a1-16f7aadd6284 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:24.871437 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c9fe7-35a5-4146-9736-dcd7a81060ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:24.872923 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c9fe7-35a5-4146-9736-dcd7a81060ca None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:24.873428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c9fe7-35a5-4146-9736-dcd7a81060ca None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:24.873922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c9fe7-35a5-4146-9736-dcd7a81060ca None admin] PATH_INFO: `/v3/projects/cabe5626f5294da3a96fced92def22c6` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:24.876839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c9fe7-35a5-4146-9736-dcd7a81060ca None admin] RBAC: Authorizing `identity:get_project(project_id=cabe5626f5294da3a96fced92def22c6)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:24.879675 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c9fe7-35a5-4146-9736-dcd7a81060ca None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:24.882954 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 572/1129] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:52:24 2022] GET /identity/v3/projects/cabe5626f5294da3a96fced92def22c6 => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:24.889248 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971c8de-7b65-427a-9290-e8d99dc9fa0a None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:24.962953 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:25.003282 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971c8de-7b65-427a-9290-e8d99dc9fa0a None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:25.039545 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91ea6b0-f9f9-43d1-b061-c7b70ebf5750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:25.040268 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ea6b0-f9f9-43d1-b061-c7b70ebf5750 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:25.040592 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ea6b0-f9f9-43d1-b061-c7b70ebf5750 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:25.040911 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ea6b0-f9f9-43d1-b061-c7b70ebf5750 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:25.041792 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91ea6b0-f9f9-43d1-b061-c7b70ebf5750 None admin] RBAC: Adding query filter params (role.id=e731efeb1d3c407fb58cf64a02d95796, scope.project.id=cabe5626f5294da3a96fced92def22c6, user.id=7dd07d2e54474c0c84b45fb6227e7e33) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:25.042147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91ea6b0-f9f9-43d1-b061-c7b70ebf5750 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:25.043684 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91ea6b0-f9f9-43d1-b061-c7b70ebf5750 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:25.050035 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 558/1130] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:52:24 2022] GET /identity/v3/role_assignments?user.id=7dd07d2e54474c0c84b45fb6227e7e33&scope.project.id=cabe5626f5294da3a96fced92def22c6&role.id=e731efeb1d3c407fb58cf64a02d95796 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:27.334901 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20da7c-10ab-4649-95e4-111b2394b08c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:27.335424 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20da7c-10ab-4649-95e4-111b2394b08c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:27.335535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20da7c-10ab-4649-95e4-111b2394b08c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:27.336825 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 573/1131] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:52:27.341974 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d940716-7800-4338-8f7f-4ad8aa474781 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:27.342411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d940716-7800-4338-8f7f-4ad8aa474781 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:27.342567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d940716-7800-4338-8f7f-4ad8aa474781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:27.359487 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-6d940716-7800-4338-8f7f-4ad8aa474781 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:27.360830 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-6d940716-7800-4338-8f7f-4ad8aa474781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:27.428688 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 559/1132] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:52:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:52:27.452229 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20da7c-10ab-4649-95e4-111b2394b08c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:27.467430 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:27.564611 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20da7c-10ab-4649-95e4-111b2394b08c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:27.599137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b88d0e6-ecbe-4b6e-b290-7c6a6275ca35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:27.599938 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88d0e6-ecbe-4b6e-b290-7c6a6275ca35 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:27.600222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88d0e6-ecbe-4b6e-b290-7c6a6275ca35 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:27.600499 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88d0e6-ecbe-4b6e-b290-7c6a6275ca35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:27.601673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b88d0e6-ecbe-4b6e-b290-7c6a6275ca35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:27.603143 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b88d0e6-ecbe-4b6e-b290-7c6a6275ca35 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:27.604776 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 574/1133] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:52:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:27.611166 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d940716-7800-4338-8f7f-4ad8aa474781 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:27.673799 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:27.713524 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d940716-7800-4338-8f7f-4ad8aa474781 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:27.748110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabb57e-15d4-49c2-a0ad-37dd736ddfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:27.748773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabb57e-15d4-49c2-a0ad-37dd736ddfda None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:27.749057 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabb57e-15d4-49c2-a0ad-37dd736ddfda None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:27.749332 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabb57e-15d4-49c2-a0ad-37dd736ddfda None admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:27.750051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dabb57e-15d4-49c2-a0ad-37dd736ddfda None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:27.752404 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dabb57e-15d4-49c2-a0ad-37dd736ddfda None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:27.778237 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 560/1134] 10.222.0.76 () {60 vars in 1141 bytes} [Thu May 26 17:52:27 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:52:30.086902 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b70078-1dc4-47eb-8aeb-4b832185a254 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:30.087462 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b70078-1dc4-47eb-8aeb-4b832185a254 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:30.087566 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b70078-1dc4-47eb-8aeb-4b832185a254 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:30.088855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 575/1135] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:52:30.094234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a37cd-d355-4de9-ab5e-47568cce0f7c None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:30.094687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a37cd-d355-4de9-ab5e-47568cce0f7c None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:30.094806 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a37cd-d355-4de9-ab5e-47568cce0f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:30.112736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-aa2a37cd-d355-4de9-ab5e-47568cce0f7c None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:30.113926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-aa2a37cd-d355-4de9-ab5e-47568cce0f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:30.181377 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 561/1136] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:52:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:52:30.206606 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b70078-1dc4-47eb-8aeb-4b832185a254 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:30.288370 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:30.309010 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b70078-1dc4-47eb-8aeb-4b832185a254 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:30.343387 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675afe21-b888-469e-b2e0-6d7abe80b907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:30.344102 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675afe21-b888-469e-b2e0-6d7abe80b907 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:30.344540 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675afe21-b888-469e-b2e0-6d7abe80b907 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:30.345152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675afe21-b888-469e-b2e0-6d7abe80b907 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:30.346111 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675afe21-b888-469e-b2e0-6d7abe80b907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:30.347570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675afe21-b888-469e-b2e0-6d7abe80b907 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:30.349207 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 576/1137] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:52:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:30.355062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a37cd-d355-4de9-ab5e-47568cce0f7c None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:30.400472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:30.457083 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a37cd-d355-4de9-ab5e-47568cce0f7c None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:30.491476 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dab04f-0ad2-49fb-b988-749bbf867e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:30.492150 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dab04f-0ad2-49fb-b988-749bbf867e10 None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:30.492437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dab04f-0ad2-49fb-b988-749bbf867e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:30.492712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dab04f-0ad2-49fb-b988-749bbf867e10 None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:30.493446 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dab04f-0ad2-49fb-b988-749bbf867e10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:30.494940 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dab04f-0ad2-49fb-b988-749bbf867e10 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:30.514478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 562/1138] 10.222.0.76 () {60 vars in 1132 bytes} [Thu May 26 17:52:30 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:52:32.833036 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a7528-0f17-403c-be28-eeeced0267cd None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:32.833620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a7528-0f17-403c-be28-eeeced0267cd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:32.833620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a7528-0f17-403c-be28-eeeced0267cd None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:32.835067 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 577/1139] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:52:32.841173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31548d1f-0292-49b2-93c2-e3ecdb308f28 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:32.841707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31548d1f-0292-49b2-93c2-e3ecdb308f28 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:32.841813 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31548d1f-0292-49b2-93c2-e3ecdb308f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:32.861037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-31548d1f-0292-49b2-93c2-e3ecdb308f28 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:32.862318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-31548d1f-0292-49b2-93c2-e3ecdb308f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:32.935109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 563/1140] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:52:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:52:32.965958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a7528-0f17-403c-be28-eeeced0267cd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:33.071809 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a7528-0f17-403c-be28-eeeced0267cd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:33.107703 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ffd252-637d-4e4c-ae0b-58002b2a3ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:33.108414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffd252-637d-4e4c-ae0b-58002b2a3ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:33.108696 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffd252-637d-4e4c-ae0b-58002b2a3ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:33.108974 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffd252-637d-4e4c-ae0b-58002b2a3ad1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:33.119986 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ffd252-637d-4e4c-ae0b-58002b2a3ad1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:52:33.127988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-88ffd252-637d-4e4c-ae0b-58002b2a3ad1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:52:33.127988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:52:33.127988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:52:33.127988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:52:33.127988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:52:33.127988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:52:33.127988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:52:33.127988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:52:33.127988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:52:33.127988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:52:33.127988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:52:33.127988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:52:33.127988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:52:33.127988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:52:33.127988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:52:33.127988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:52:33.129238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:52:33.129238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:52:33.129238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:52:33.129238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:52:33.129238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:52:33.129238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:52:33.129238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:52:33.129238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:52:33.129238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:52:33.129238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:52:33.129238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:52:33.129238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:52:33.129238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:52:33.129238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:52:33.129238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:52:33.129238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:52:33.134890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:52:33.134890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:52:33.134890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:52:33.134890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:52:33.134890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:52:33.134890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:52:33.134890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:52:33.134890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:52:33.134890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:52:33.134890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:52:33.134890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:52:33.134890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:52:33.134890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 578/1141] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:52:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:52:33.135872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31548d1f-0292-49b2-93c2-e3ecdb308f28 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:33.167654 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:33.239828 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31548d1f-0292-49b2-93c2-e3ecdb308f28 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:33.275262 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1f38c-4754-43cc-b5f6-fdf68f65eb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:33.275934 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1f38c-4754-43cc-b5f6-fdf68f65eb11 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:33.276216 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1f38c-4754-43cc-b5f6-fdf68f65eb11 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:33.276490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1f38c-4754-43cc-b5f6-fdf68f65eb11 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:33.277250 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1f38c-4754-43cc-b5f6-fdf68f65eb11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:33.277555 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1f38c-4754-43cc-b5f6-fdf68f65eb11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:33.278981 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1f38c-4754-43cc-b5f6-fdf68f65eb11 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:33.289055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 564/1142] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:52:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:33.295154 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ffd252-637d-4e4c-ae0b-58002b2a3ad1 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:33.308968 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:33.402944 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ffd252-637d-4e4c-ae0b-58002b2a3ad1 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:33.438644 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26c02e-4945-41b1-be5c-14e28694600a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:33.439642 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26c02e-4945-41b1-be5c-14e28694600a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:33.439924 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26c02e-4945-41b1-be5c-14e28694600a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:33.440199 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26c02e-4945-41b1-be5c-14e28694600a None admin] PATH_INFO: `/v3/users/d589312b67dd4a0e850e610ff0cb5e9e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:33.454978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26c02e-4945-41b1-be5c-14e28694600a None admin] RBAC: Authorizing `identity:get_user(user_id=d589312b67dd4a0e850e610ff0cb5e9e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:33.456450 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26c02e-4945-41b1-be5c-14e28694600a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:33.458180 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 579/1143] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:52:33 2022] GET /identity/v3/users/d589312b67dd4a0e850e610ff0cb5e9e => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:33.464678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1f38c-4754-43cc-b5f6-fdf68f65eb11 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:33.564459 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:33.574917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1f38c-4754-43cc-b5f6-fdf68f65eb11 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:33.611844 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0620ae75-a4b3-423e-8dfc-9c4278260a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:33.612670 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620ae75-a4b3-423e-8dfc-9c4278260a07 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:33.612958 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620ae75-a4b3-423e-8dfc-9c4278260a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:33.613238 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620ae75-a4b3-423e-8dfc-9c4278260a07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:33.683986 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0620ae75-a4b3-423e-8dfc-9c4278260a07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:33.685362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0620ae75-a4b3-423e-8dfc-9c4278260a07 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:33.777535 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:33.790731 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 565/1144] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:52:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:52:33.796713 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26c02e-4945-41b1-be5c-14e28694600a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:33.864930 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:33.904056 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26c02e-4945-41b1-be5c-14e28694600a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:33.941148 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54910ef6-9baf-417d-b729-0b45dd43aab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:33.941928 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54910ef6-9baf-417d-b729-0b45dd43aab8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:33.942243 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54910ef6-9baf-417d-b729-0b45dd43aab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:33.942528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54910ef6-9baf-417d-b729-0b45dd43aab8 None admin] PATH_INFO: `/v3/projects/411a106791ec4076a64bafd65a544209` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:33.943717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54910ef6-9baf-417d-b729-0b45dd43aab8 None admin] RBAC: Authorizing `identity:get_project(project_id=411a106791ec4076a64bafd65a544209)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:33.945152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54910ef6-9baf-417d-b729-0b45dd43aab8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:33.946910 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 580/1145] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:52:33 2022] GET /identity/v3/projects/411a106791ec4076a64bafd65a544209 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:33.953145 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0620ae75-a4b3-423e-8dfc-9c4278260a07 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:34.054848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0620ae75-a4b3-423e-8dfc-9c4278260a07 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:34.088845 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74041050-80f0-4474-9893-5856443ce0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:34.089502 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74041050-80f0-4474-9893-5856443ce0fd None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:34.089787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74041050-80f0-4474-9893-5856443ce0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:34.090066 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74041050-80f0-4474-9893-5856443ce0fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:34.090935 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74041050-80f0-4474-9893-5856443ce0fd None admin] RBAC: Adding query filter params (role.id=4fa8f06085c741a5b3cf64e63cc0754f, scope.project.id=411a106791ec4076a64bafd65a544209, user.id=d589312b67dd4a0e850e610ff0cb5e9e) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:34.091252 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74041050-80f0-4474-9893-5856443ce0fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:34.092695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74041050-80f0-4474-9893-5856443ce0fd None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:34.098839 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 566/1146] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:52:33 2022] GET /identity/v3/role_assignments?user.id=d589312b67dd4a0e850e610ff0cb5e9e&scope.project.id=411a106791ec4076a64bafd65a544209&role.id=4fa8f06085c741a5b3cf64e63cc0754f => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:36.349823 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70665464-7433-4daa-9e74-62f8f000e42c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:36.350371 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70665464-7433-4daa-9e74-62f8f000e42c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:36.350462 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70665464-7433-4daa-9e74-62f8f000e42c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:36.351799 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 581/1147] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:52:36.357383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad81df7-4425-42e6-ad44-9fe4e8669178 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:36.357854 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad81df7-4425-42e6-ad44-9fe4e8669178 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:36.358020 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad81df7-4425-42e6-ad44-9fe4e8669178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:36.375529 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-7ad81df7-4425-42e6-ad44-9fe4e8669178 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:36.377251 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-7ad81df7-4425-42e6-ad44-9fe4e8669178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:36.430518 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:36.447551 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 567/1148] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:52:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:52:36.471183 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70665464-7433-4daa-9e74-62f8f000e42c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:36.578058 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70665464-7433-4daa-9e74-62f8f000e42c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:36.612404 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35695046-aa32-425b-885d-f5a6221f249c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:36.613194 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35695046-aa32-425b-885d-f5a6221f249c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:36.613477 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35695046-aa32-425b-885d-f5a6221f249c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:36.613754 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35695046-aa32-425b-885d-f5a6221f249c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:36.615658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:36.625283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35695046-aa32-425b-885d-f5a6221f249c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:52:36.635226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-35695046-aa32-425b-885d-f5a6221f249c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:52:36.635226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:52:36.635226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:52:36.635226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:52:36.635226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:52:36.635226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:52:36.635226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:52:36.635226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:52:36.635226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:52:36.635226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:52:36.635226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:52:36.635226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:52:36.635226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:52:36.635226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:52:36.635226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:52:36.635226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:52:36.636360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:52:36.636360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:52:36.636360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:52:36.636360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:52:36.636360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:52:36.636360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:52:36.636360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:52:36.636360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:52:36.636360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:52:36.636360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:52:36.636360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:52:36.636360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:52:36.636360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:52:36.636360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:52:36.636360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:52:36.636360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:52:36.637533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:52:36.637533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:52:36.637533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:52:36.637533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:52:36.637533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:52:36.637533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:52:36.637533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:52:36.637533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:52:36.637533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:52:36.637533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:52:36.637533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:52:36.637533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:52:36.637533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 582/1149] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:52:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:52:36.644087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad81df7-4425-42e6-ad44-9fe4e8669178 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:36.742832 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad81df7-4425-42e6-ad44-9fe4e8669178 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:36.780012 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb8095-e85b-488b-904e-019797c65911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:36.780696 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb8095-e85b-488b-904e-019797c65911 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:36.780981 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb8095-e85b-488b-904e-019797c65911 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:36.781259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb8095-e85b-488b-904e-019797c65911 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:36.782020 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb8095-e85b-488b-904e-019797c65911 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:36.782363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb8095-e85b-488b-904e-019797c65911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:36.783868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb8095-e85b-488b-904e-019797c65911 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:36.793002 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 568/1150] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:52:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:36.799129 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35695046-aa32-425b-885d-f5a6221f249c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:36.901237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35695046-aa32-425b-885d-f5a6221f249c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:36.936396 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e74de1-fe6d-4314-b8bc-3b9ac9398c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:36.937266 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e74de1-fe6d-4314-b8bc-3b9ac9398c7f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:36.937554 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e74de1-fe6d-4314-b8bc-3b9ac9398c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:36.937833 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e74de1-fe6d-4314-b8bc-3b9ac9398c7f None admin] PATH_INFO: `/v3/users/d589312b67dd4a0e850e610ff0cb5e9e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:36.939969 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e74de1-fe6d-4314-b8bc-3b9ac9398c7f None admin] RBAC: Authorizing `identity:get_user(user_id=d589312b67dd4a0e850e610ff0cb5e9e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:36.941489 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e74de1-fe6d-4314-b8bc-3b9ac9398c7f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:36.943169 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 583/1151] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:52:36 2022] GET /identity/v3/users/d589312b67dd4a0e850e610ff0cb5e9e => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:52:36.948912 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb8095-e85b-488b-904e-019797c65911 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:36.997648 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:37.050638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb8095-e85b-488b-904e-019797c65911 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:37.085476 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1473c2c-c985-4391-93f1-04d101d81012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:37.086374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1473c2c-c985-4391-93f1-04d101d81012 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:37.086661 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1473c2c-c985-4391-93f1-04d101d81012 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:37.086941 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1473c2c-c985-4391-93f1-04d101d81012 None admin] PATH_INFO: `/v3/projects/411a106791ec4076a64bafd65a544209` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:37.088123 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1473c2c-c985-4391-93f1-04d101d81012 None admin] RBAC: Authorizing `identity:get_project(project_id=411a106791ec4076a64bafd65a544209)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:37.089577 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1473c2c-c985-4391-93f1-04d101d81012 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:37.091304 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 569/1152] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:52:36 2022] GET /identity/v3/projects/411a106791ec4076a64bafd65a544209 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:37.097271 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e74de1-fe6d-4314-b8bc-3b9ac9398c7f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:37.162629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:37.204026 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e74de1-fe6d-4314-b8bc-3b9ac9398c7f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:37.239039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878153af-13c1-4b16-b98f-9806782422ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:37.239937 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878153af-13c1-4b16-b98f-9806782422ca None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:37.240224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878153af-13c1-4b16-b98f-9806782422ca None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:37.240502 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878153af-13c1-4b16-b98f-9806782422ca None admin] PATH_INFO: `/v3/projects/411a106791ec4076a64bafd65a544209/users/d589312b67dd4a0e850e610ff0cb5e9e/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:37.243285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878153af-13c1-4b16-b98f-9806782422ca None admin] RBAC: Authorizing `identity:create_grant(project_id=411a106791ec4076a64bafd65a544209, user_id=d589312b67dd4a0e850e610ff0cb5e9e, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:37.244728 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878153af-13c1-4b16-b98f-9806782422ca None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:37.258550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 584/1153] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:52:37 2022] PUT /identity/v3/projects/411a106791ec4076a64bafd65a544209/users/d589312b67dd4a0e850e610ff0cb5e9e/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:52:39.560231 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fb5fd-bf8a-4bd6-85d3-46313bf8a092 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:39.560790 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fb5fd-bf8a-4bd6-85d3-46313bf8a092 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:39.560790 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fb5fd-bf8a-4bd6-85d3-46313bf8a092 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:39.562164 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 570/1154] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:52:39.567973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cc8f2-3417-41e1-9aba-f2805c253c38 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:39.568498 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cc8f2-3417-41e1-9aba-f2805c253c38 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:39.568498 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cc8f2-3417-41e1-9aba-f2805c253c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:39.585503 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8c9cc8f2-3417-41e1-9aba-f2805c253c38 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:39.586734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8c9cc8f2-3417-41e1-9aba-f2805c253c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:39.656152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 585/1155] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:52:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:52:39.681883 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fb5fd-bf8a-4bd6-85d3-46313bf8a092 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:39.793566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fb5fd-bf8a-4bd6-85d3-46313bf8a092 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:39.812772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:39.832257 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46be2b-db57-4735-ab20-d3e1891e0fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:39.833060 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46be2b-db57-4735-ab20-d3e1891e0fae None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:39.833340 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46be2b-db57-4735-ab20-d3e1891e0fae None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:39.833615 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46be2b-db57-4735-ab20-d3e1891e0fae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:39.844855 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46be2b-db57-4735-ab20-d3e1891e0fae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:52:39.853007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-7b46be2b-db57-4735-ab20-d3e1891e0fae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:52:39.853007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:52:39.853007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:52:39.853007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:52:39.853007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:52:39.853007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:52:39.853007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:52:39.853007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:52:39.853007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:52:39.853007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:52:39.853007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:52:39.853007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:52:39.853007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:52:39.853007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:52:39.853007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:52:39.853007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:52:39.854241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:52:39.854241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:52:39.854241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:52:39.854241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:52:39.854241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:52:39.854241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:52:39.854241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:52:39.854241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:52:39.854241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:52:39.854241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:52:39.854241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:52:39.854241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:52:39.854241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:52:39.854241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:52:39.854241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:52:39.854241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:52:39.855347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:52:39.855347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:52:39.855347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:52:39.855347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:52:39.855347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:52:39.855347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:52:39.855347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:52:39.855347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:52:39.855347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:52:39.855347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:52:39.855347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:52:39.855347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:52:39.855347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 571/1156] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:52:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:52:39.861094 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9cc8f2-3417-41e1-9aba-f2805c253c38 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:39.913965 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:39.962295 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9cc8f2-3417-41e1-9aba-f2805c253c38 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:39.997198 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936055a-2b89-418f-ae42-fc9754a0ddbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:39.997870 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936055a-2b89-418f-ae42-fc9754a0ddbb None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:39.998148 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936055a-2b89-418f-ae42-fc9754a0ddbb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:39.998455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936055a-2b89-418f-ae42-fc9754a0ddbb None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:39.999201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a936055a-2b89-418f-ae42-fc9754a0ddbb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:39.999502 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a936055a-2b89-418f-ae42-fc9754a0ddbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:40.002044 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a936055a-2b89-418f-ae42-fc9754a0ddbb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:40.010493 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 586/1157] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:52:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:40.016353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46be2b-db57-4735-ab20-d3e1891e0fae None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:40.119576 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46be2b-db57-4735-ab20-d3e1891e0fae None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:40.155362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69aea9-4114-4cd8-8982-51acf86fc80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:40.156235 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69aea9-4114-4cd8-8982-51acf86fc80c None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:40.156513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69aea9-4114-4cd8-8982-51acf86fc80c None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:40.156780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69aea9-4114-4cd8-8982-51acf86fc80c None admin] PATH_INFO: `/v3/users/d589312b67dd4a0e850e610ff0cb5e9e` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:40.158253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69aea9-4114-4cd8-8982-51acf86fc80c None admin] RBAC: Authorizing `identity:get_user(user_id=d589312b67dd4a0e850e610ff0cb5e9e)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:40.159695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69aea9-4114-4cd8-8982-51acf86fc80c None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:40.161633 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 572/1158] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:52:40 2022] GET /identity/v3/users/d589312b67dd4a0e850e610ff0cb5e9e => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:40.167444 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936055a-2b89-418f-ae42-fc9754a0ddbb None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:40.284307 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936055a-2b89-418f-ae42-fc9754a0ddbb None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:40.299404 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:40.320883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf421148-c264-416a-afed-11570d061f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:40.321546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf421148-c264-416a-afed-11570d061f17 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:40.321826 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf421148-c264-416a-afed-11570d061f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:40.322096 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf421148-c264-416a-afed-11570d061f17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:40.390954 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf421148-c264-416a-afed-11570d061f17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:40.392836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf421148-c264-416a-afed-11570d061f17 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:40.494883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 587/1159] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:52:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:52:40.501299 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69aea9-4114-4cd8-8982-51acf86fc80c None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:40.550458 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:40.607869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69aea9-4114-4cd8-8982-51acf86fc80c None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:40.647205 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a7aa8-052a-40cb-929a-fd14b21bfdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:40.648108 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a7aa8-052a-40cb-929a-fd14b21bfdd3 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:40.648399 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a7aa8-052a-40cb-929a-fd14b21bfdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:40.648679 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a7aa8-052a-40cb-929a-fd14b21bfdd3 None admin] PATH_INFO: `/v3/projects/411a106791ec4076a64bafd65a544209` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:40.649915 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655a7aa8-052a-40cb-929a-fd14b21bfdd3 None admin] RBAC: Authorizing `identity:get_project(project_id=411a106791ec4076a64bafd65a544209)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:40.651417 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655a7aa8-052a-40cb-929a-fd14b21bfdd3 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:40.653231 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 573/1160] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:52:40 2022] GET /identity/v3/projects/411a106791ec4076a64bafd65a544209 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:40.659912 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf421148-c264-416a-afed-11570d061f17 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:40.677213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:40.763763 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf421148-c264-416a-afed-11570d061f17 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:40.800035 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665431f0-da1b-4a14-b277-b680246de9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:40.800695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665431f0-da1b-4a14-b277-b680246de9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:40.800982 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665431f0-da1b-4a14-b277-b680246de9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:40.801265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665431f0-da1b-4a14-b277-b680246de9b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:40.802102 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665431f0-da1b-4a14-b277-b680246de9b7 None admin] RBAC: Adding query filter params (role.id=4fa8f06085c741a5b3cf64e63cc0754f, scope.project.id=411a106791ec4076a64bafd65a544209, user.id=d589312b67dd4a0e850e610ff0cb5e9e) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:40.802449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665431f0-da1b-4a14-b277-b680246de9b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:40.803937 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665431f0-da1b-4a14-b277-b680246de9b7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:40.810402 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 588/1161] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:52:40 2022] GET /identity/v3/role_assignments?user.id=d589312b67dd4a0e850e610ff0cb5e9e&scope.project.id=411a106791ec4076a64bafd65a544209&role.id=4fa8f06085c741a5b3cf64e63cc0754f => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:43.105723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb9892-39bf-43a6-b6bc-de7f3b9c65aa None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:43.106305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb9892-39bf-43a6-b6bc-de7f3b9c65aa None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:43.106305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb9892-39bf-43a6-b6bc-de7f3b9c65aa None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:43.107672 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 574/1162] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:52:43.113109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d8efc-5630-4cd2-b08f-10529d6b1e2f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:43.113541 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d8efc-5630-4cd2-b08f-10529d6b1e2f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:43.113677 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d8efc-5630-4cd2-b08f-10529d6b1e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:43.130912 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-945d8efc-5630-4cd2-b08f-10529d6b1e2f None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:43.132110 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-945d8efc-5630-4cd2-b08f-10529d6b1e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:43.200699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 589/1163] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:52:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:52:43.225199 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb9892-39bf-43a6-b6bc-de7f3b9c65aa None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:43.332878 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb9892-39bf-43a6-b6bc-de7f3b9c65aa None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:43.347234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:43.369705 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70994bd-54f1-4927-b787-0f0fcb326e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:43.370294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70994bd-54f1-4927-b787-0f0fcb326e47 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:43.370584 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70994bd-54f1-4927-b787-0f0fcb326e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:43.370866 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70994bd-54f1-4927-b787-0f0fcb326e47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:43.446802 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70994bd-54f1-4927-b787-0f0fcb326e47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:43.448298 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70994bd-54f1-4927-b787-0f0fcb326e47 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:43.553516 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 575/1164] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:52:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:52:43.559589 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d8efc-5630-4cd2-b08f-10529d6b1e2f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:43.569869 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:43.662595 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d8efc-5630-4cd2-b08f-10529d6b1e2f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:43.700507 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed9773-4c18-4db3-bfd1-108e4fe665c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:43.701321 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed9773-4c18-4db3-bfd1-108e4fe665c3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:43.701607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed9773-4c18-4db3-bfd1-108e4fe665c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:43.701884 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed9773-4c18-4db3-bfd1-108e4fe665c3 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:43.710457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed9773-4c18-4db3-bfd1-108e4fe665c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:43.711946 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed9773-4c18-4db3-bfd1-108e4fe665c3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:43.728316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-01ed9773-4c18-4db3-bfd1-108e4fe665c3 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 26 17:52:43.728316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:52:43.728316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 26 17:52:43.728316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 26 17:52:43.728316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:52:43.728316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:52:43.728316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:52:43.728316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:52:43.728316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 26 17:52:43.728316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:52:43.728316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:52:43.728316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:52:43.728316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:52:43.728316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:52:43.728316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:52:43.728316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:52:43.728316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:52:43.730596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:52:43.730596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:52:43.730596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:52:43.730596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:52:43.730596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:52:43.730596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:52:43.730596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:52:43.730596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:52:43.730596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:52:43.730596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:52:43.730596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:52:43.730596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:52:43.730596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:52:43.730596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:52:43.730596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:52:43.730596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:52:43.732556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:52:43.732556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:52:43.732556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:52:43.732556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:52:43.732556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:52:43.732556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:52:43.732556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:52:43.732556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:52:43.732556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:52:43.732556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:52:43.732556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 26 17:52:43.732556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:52:43.732556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 26 17:52:43.732556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:52:43.732556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 590/1165] 10.222.0.76 () {56 vars in 1117 bytes} [Thu May 26 17:52:43 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:52:43.738027 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70994bd-54f1-4927-b787-0f0fcb326e47 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:43.756159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:43.843612 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70994bd-54f1-4927-b787-0f0fcb326e47 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:43.878514 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20612521-2782-4e73-8b78-ee19793378b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:43.879179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20612521-2782-4e73-8b78-ee19793378b4 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:43.879465 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20612521-2782-4e73-8b78-ee19793378b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:43.879745 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20612521-2782-4e73-8b78-ee19793378b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:43.880472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20612521-2782-4e73-8b78-ee19793378b4 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:43.880783 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20612521-2782-4e73-8b78-ee19793378b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:43.882161 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20612521-2782-4e73-8b78-ee19793378b4 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:43.889915 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 576/1166] 10.222.0.76 () {56 vars in 1115 bytes} [Thu May 26 17:52:43 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:46.202637 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6863a0-4189-4a50-8d02-7bc439ff5a1c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:46.203159 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6863a0-4189-4a50-8d02-7bc439ff5a1c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:46.203271 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6863a0-4189-4a50-8d02-7bc439ff5a1c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:46.204724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 591/1167] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:52:46.210384 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061522f-f106-41a7-bf0d-168854b79c60 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:46.210849 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061522f-f106-41a7-bf0d-168854b79c60 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:46.210941 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061522f-f106-41a7-bf0d-168854b79c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:46.229740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-f061522f-f106-41a7-bf0d-168854b79c60 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:46.231006 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f061522f-f106-41a7-bf0d-168854b79c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:46.302006 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 577/1168] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:52:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:52:46.326100 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6863a0-4189-4a50-8d02-7bc439ff5a1c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:46.397781 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:46.434470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6863a0-4189-4a50-8d02-7bc439ff5a1c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:46.468537 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd73a56-05b7-4cba-bcc3-c1ead0da2c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:46.469919 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd73a56-05b7-4cba-bcc3-c1ead0da2c1e None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:46.470636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd73a56-05b7-4cba-bcc3-c1ead0da2c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:46.470911 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd73a56-05b7-4cba-bcc3-c1ead0da2c1e None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:46.471660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd73a56-05b7-4cba-bcc3-c1ead0da2c1e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:46.473038 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd73a56-05b7-4cba-bcc3-c1ead0da2c1e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:46.489259 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 592/1169] 10.222.0.76 () {60 vars in 1138 bytes} [Thu May 26 17:52:46 2022] POST /identity/v3/domains => generated 267 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:52:48.768122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45cd41f-9342-4fd6-9500-2aad6ba1d8f2 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:48.768684 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45cd41f-9342-4fd6-9500-2aad6ba1d8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:48.768684 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45cd41f-9342-4fd6-9500-2aad6ba1d8f2 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:48.770039 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 578/1170] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:52:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:52:48.775436 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597d391-70b7-44dc-aa3c-cac5f6af11f0 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:48.775879 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597d391-70b7-44dc-aa3c-cac5f6af11f0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:48.776005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597d391-70b7-44dc-aa3c-cac5f6af11f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:48.793388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2597d391-70b7-44dc-aa3c-cac5f6af11f0 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:48.794626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2597d391-70b7-44dc-aa3c-cac5f6af11f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:48.862742 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 593/1171] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:52:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:52:48.887331 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45cd41f-9342-4fd6-9500-2aad6ba1d8f2 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:48.896386 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:48.990888 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45cd41f-9342-4fd6-9500-2aad6ba1d8f2 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:49.028479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a1e84-6c8b-483a-a8c0-0a2c9371b66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:49.029211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a1e84-6c8b-483a-a8c0-0a2c9371b66b None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:49.029493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a1e84-6c8b-483a-a8c0-0a2c9371b66b None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:49.029764 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a1e84-6c8b-483a-a8c0-0a2c9371b66b None admin] PATH_INFO: `/v3/domains/eda516accd6d4e4089f262bb9d8adb8c` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:49.041015 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a1e84-6c8b-483a-a8c0-0a2c9371b66b None admin] RBAC: Authorizing `identity:get_domain(domain_id=eda516accd6d4e4089f262bb9d8adb8c)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:49.042482 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a1e84-6c8b-483a-a8c0-0a2c9371b66b None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:49.044113 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 579/1172] 10.222.0.76 () {56 vars in 1183 bytes} [Thu May 26 17:52:48 2022] GET /identity/v3/domains/eda516accd6d4e4089f262bb9d8adb8c => generated 267 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:49.050045 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597d391-70b7-44dc-aa3c-cac5f6af11f0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:49.106235 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:49.147738 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597d391-70b7-44dc-aa3c-cac5f6af11f0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:49.182184 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d67bd-7c66-4968-a997-82a44b6e1705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:49.182866 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d67bd-7c66-4968-a997-82a44b6e1705 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:49.183151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d67bd-7c66-4968-a997-82a44b6e1705 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:49.183426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d67bd-7c66-4968-a997-82a44b6e1705 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:49.184134 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d67bd-7c66-4968-a997-82a44b6e1705 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:49.185566 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d67bd-7c66-4968-a997-82a44b6e1705 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:49.211702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 594/1173] 10.222.0.76 () {60 vars in 1141 bytes} [Thu May 26 17:52:49 2022] POST /identity/v3/projects => generated 361 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:52:51.464819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfff11-6a76-414d-8266-9e89157b0f28 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:51.465391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfff11-6a76-414d-8266-9e89157b0f28 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:51.465391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfff11-6a76-414d-8266-9e89157b0f28 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:51.466719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 580/1174] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:52:51.473299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976c7b1-0b3a-4950-a28a-17ffd84cd5b9 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:51.473888 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976c7b1-0b3a-4950-a28a-17ffd84cd5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:51.474171 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976c7b1-0b3a-4950-a28a-17ffd84cd5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:51.491756 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0976c7b1-0b3a-4950-a28a-17ffd84cd5b9 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:51.492953 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0976c7b1-0b3a-4950-a28a-17ffd84cd5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:51.562756 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 595/1175] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:52:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:52:51.587877 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfff11-6a76-414d-8266-9e89157b0f28 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:51.655537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:51.697023 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfff11-6a76-414d-8266-9e89157b0f28 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:51.735293 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173f65f-7498-480e-b287-67d36cc13503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:51.736093 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173f65f-7498-480e-b287-67d36cc13503 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:51.736377 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173f65f-7498-480e-b287-67d36cc13503 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:51.736657 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173f65f-7498-480e-b287-67d36cc13503 None admin] PATH_INFO: `/v3/domains/eda516accd6d4e4089f262bb9d8adb8c` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:51.737970 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173f65f-7498-480e-b287-67d36cc13503 None admin] RBAC: Authorizing `identity:get_domain(domain_id=eda516accd6d4e4089f262bb9d8adb8c)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:51.739624 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173f65f-7498-480e-b287-67d36cc13503 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:51.741314 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 581/1176] 10.222.0.76 () {56 vars in 1183 bytes} [Thu May 26 17:52:51 2022] GET /identity/v3/domains/eda516accd6d4e4089f262bb9d8adb8c => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:51.747509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976c7b1-0b3a-4950-a28a-17ffd84cd5b9 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:51.807193 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:51.870164 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976c7b1-0b3a-4950-a28a-17ffd84cd5b9 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:51.906201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324fe8f-8b63-4281-9910-f72b01e795ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:51.908289 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324fe8f-8b63-4281-9910-f72b01e795ad None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:51.908586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324fe8f-8b63-4281-9910-f72b01e795ad None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:51.908854 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324fe8f-8b63-4281-9910-f72b01e795ad None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:51.909599 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324fe8f-8b63-4281-9910-f72b01e795ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:51.911287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324fe8f-8b63-4281-9910-f72b01e795ad None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:51.933602 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 596/1177] 10.222.0.76 () {60 vars in 1132 bytes} [Thu May 26 17:52:51 2022] POST /identity/v3/users => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:52:54.278732 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2e93a-0fc1-4708-961f-5d3dce587bd2 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:54.279294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2e93a-0fc1-4708-961f-5d3dce587bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:54.279294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2e93a-0fc1-4708-961f-5d3dce587bd2 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:54.280688 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 582/1178] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:52:54.287309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a28fe3-a6bd-44cd-847d-2de75ae9b368 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:54.287876 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a28fe3-a6bd-44cd-847d-2de75ae9b368 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:54.287983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a28fe3-a6bd-44cd-847d-2de75ae9b368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:54.306964 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c1a28fe3-a6bd-44cd-847d-2de75ae9b368 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:54.308164 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c1a28fe3-a6bd-44cd-847d-2de75ae9b368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:54.384009 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 597/1179] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:52:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:52:54.408237 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2e93a-0fc1-4708-961f-5d3dce587bd2 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:54.542018 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2e93a-0fc1-4708-961f-5d3dce587bd2 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:54.582991 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d99a0-9dcb-4856-b8cc-0878b0fa83ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:54.583825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d99a0-9dcb-4856-b8cc-0878b0fa83ee None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:54.584108 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d99a0-9dcb-4856-b8cc-0878b0fa83ee None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:54.584383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d99a0-9dcb-4856-b8cc-0878b0fa83ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:54.587050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:54.599149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d99a0-9dcb-4856-b8cc-0878b0fa83ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:52:54.607036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-662d99a0-9dcb-4856-b8cc-0878b0fa83ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:52:54.607036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:52:54.607036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:52:54.607036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:52:54.607036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:52:54.607036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:52:54.607036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:52:54.607036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:52:54.607036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:52:54.607036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:52:54.607036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:52:54.607036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:52:54.607036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:52:54.607036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:52:54.607036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:52:54.607036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:52:54.608328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:52:54.608328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:52:54.608328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:52:54.608328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:52:54.608328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:52:54.608328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:52:54.608328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:52:54.608328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:52:54.608328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:52:54.608328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:52:54.608328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:52:54.608328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:52:54.608328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:52:54.608328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:52:54.608328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:52:54.608328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:52:54.609244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:52:54.609244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:52:54.609244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:52:54.609244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:52:54.609244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:52:54.609244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:52:54.609244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:52:54.609244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:52:54.609244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:52:54.609244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:52:54.609244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:52:54.609244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:52:54.609244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 583/1180] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:52:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:52:54.615669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a28fe3-a6bd-44cd-847d-2de75ae9b368 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:54.643237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:54.717692 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a28fe3-a6bd-44cd-847d-2de75ae9b368 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:54.753727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74fca6-dfee-4503-8a0a-cedd06a715db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:54.754424 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74fca6-dfee-4503-8a0a-cedd06a715db None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:54.754714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74fca6-dfee-4503-8a0a-cedd06a715db None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:54.754991 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74fca6-dfee-4503-8a0a-cedd06a715db None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:54.755735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74fca6-dfee-4503-8a0a-cedd06a715db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:54.756042 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74fca6-dfee-4503-8a0a-cedd06a715db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:54.757414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74fca6-dfee-4503-8a0a-cedd06a715db None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:54.765806 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 598/1181] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:52:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:54.771856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d99a0-9dcb-4856-b8cc-0878b0fa83ee None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:54.885988 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d99a0-9dcb-4856-b8cc-0878b0fa83ee None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:54.924226 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b55328-3596-4458-8a1a-ee69db2c7203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:54.925116 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b55328-3596-4458-8a1a-ee69db2c7203 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:54.925422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b55328-3596-4458-8a1a-ee69db2c7203 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:54.925777 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b55328-3596-4458-8a1a-ee69db2c7203 None admin] PATH_INFO: `/v3/users/8bf41df333374fb5b08553bd6ba0143e` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:54.940619 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b55328-3596-4458-8a1a-ee69db2c7203 None admin] RBAC: Authorizing `identity:get_user(user_id=8bf41df333374fb5b08553bd6ba0143e)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:54.942070 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b55328-3596-4458-8a1a-ee69db2c7203 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:54.943810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 584/1182] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:52:54 2022] GET /identity/v3/users/8bf41df333374fb5b08553bd6ba0143e => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:54.949761 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74fca6-dfee-4503-8a0a-cedd06a715db None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:55.043598 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:55.057161 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74fca6-dfee-4503-8a0a-cedd06a715db None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:55.091907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca62ea2-1a7a-4206-8ca5-db1b8a33a3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:55.092559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca62ea2-1a7a-4206-8ca5-db1b8a33a3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:55.092839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca62ea2-1a7a-4206-8ca5-db1b8a33a3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:55.093110 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca62ea2-1a7a-4206-8ca5-db1b8a33a3b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:55.175845 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca62ea2-1a7a-4206-8ca5-db1b8a33a3b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:55.177299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca62ea2-1a7a-4206-8ca5-db1b8a33a3b9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:55.267131 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:55.280663 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 599/1183] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:52:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:52:55.287124 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b55328-3596-4458-8a1a-ee69db2c7203 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:55.345852 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:55.397670 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b55328-3596-4458-8a1a-ee69db2c7203 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:55.432733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b4df3-1571-45f7-b437-3666917c2590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:55.433600 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b4df3-1571-45f7-b437-3666917c2590 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:55.433885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b4df3-1571-45f7-b437-3666917c2590 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:55.434157 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b4df3-1571-45f7-b437-3666917c2590 None admin] PATH_INFO: `/v3/projects/1cb44a82ee6b4672a8c3dfffe6936b6d` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:55.435416 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b4df3-1571-45f7-b437-3666917c2590 None admin] RBAC: Authorizing `identity:get_project(project_id=1cb44a82ee6b4672a8c3dfffe6936b6d)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:55.436869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b4df3-1571-45f7-b437-3666917c2590 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:55.439220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 585/1184] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:52:55 2022] GET /identity/v3/projects/1cb44a82ee6b4672a8c3dfffe6936b6d => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:55.445462 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca62ea2-1a7a-4206-8ca5-db1b8a33a3b9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:55.553566 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca62ea2-1a7a-4206-8ca5-db1b8a33a3b9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:55.588737 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf2789-43a0-4981-9d28-6afb169abfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:55.589519 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf2789-43a0-4981-9d28-6afb169abfca None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:55.589805 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf2789-43a0-4981-9d28-6afb169abfca None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:55.590080 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf2789-43a0-4981-9d28-6afb169abfca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:55.590935 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf2789-43a0-4981-9d28-6afb169abfca None admin] RBAC: Adding query filter params (role.id=4fa8f06085c741a5b3cf64e63cc0754f, scope.project.id=1cb44a82ee6b4672a8c3dfffe6936b6d, user.id=8bf41df333374fb5b08553bd6ba0143e) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:55.591251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf2789-43a0-4981-9d28-6afb169abfca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:55.592698 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf2789-43a0-4981-9d28-6afb169abfca None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:55.599089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 600/1185] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:52:55 2022] GET /identity/v3/role_assignments?user.id=8bf41df333374fb5b08553bd6ba0143e&scope.project.id=1cb44a82ee6b4672a8c3dfffe6936b6d&role.id=4fa8f06085c741a5b3cf64e63cc0754f => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:57.867701 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bbf5c6-456c-4db5-b07d-bd5951c7cacd None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:57.868294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bbf5c6-456c-4db5-b07d-bd5951c7cacd None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:57.868294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bbf5c6-456c-4db5-b07d-bd5951c7cacd None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:57.869665 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 586/1186] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:52:57.876102 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df591f-fc63-4177-83fa-54a2eda45b6c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:57.876668 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df591f-fc63-4177-83fa-54a2eda45b6c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:57.876906 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df591f-fc63-4177-83fa-54a2eda45b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:57.895944 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d5df591f-fc63-4177-83fa-54a2eda45b6c None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:52:57.897152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d5df591f-fc63-4177-83fa-54a2eda45b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:52:57.945729 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:57.966789 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 601/1187] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:52:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:52:57.989767 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bbf5c6-456c-4db5-b07d-bd5951c7cacd None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:58.092602 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bbf5c6-456c-4db5-b07d-bd5951c7cacd None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:58.113863 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:58.128868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f37e99-60d8-403b-89c0-9f98edf1d680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:58.129594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f37e99-60d8-403b-89c0-9f98edf1d680 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:58.129882 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f37e99-60d8-403b-89c0-9f98edf1d680 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:58.130159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f37e99-60d8-403b-89c0-9f98edf1d680 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:58.141876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f37e99-60d8-403b-89c0-9f98edf1d680 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:52:58.149562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-a1f37e99-60d8-403b-89c0-9f98edf1d680 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:52:58.149562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:52:58.149562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:52:58.149562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:52:58.149562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:52:58.149562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:52:58.149562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:52:58.149562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:52:58.149562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:52:58.149562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:52:58.149562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:52:58.149562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:52:58.149562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:52:58.149562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:52:58.149562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:52:58.149562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:52:58.150711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:52:58.150711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:52:58.150711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:52:58.150711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:52:58.150711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:52:58.150711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:52:58.150711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:52:58.150711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:52:58.150711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:52:58.150711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:52:58.150711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:52:58.150711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:52:58.150711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:52:58.150711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:52:58.150711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:52:58.150711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:52:58.151662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:52:58.151662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:52:58.151662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:52:58.151662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:52:58.151662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:52:58.151662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:52:58.151662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:52:58.151662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:52:58.151662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:52:58.151662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:52:58.151662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:52:58.151662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:52:58.151662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 587/1188] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:52:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:52:58.157463 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df591f-fc63-4177-83fa-54a2eda45b6c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:58.261230 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df591f-fc63-4177-83fa-54a2eda45b6c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:58.296827 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370fe7d-15d3-454d-96ca-692fc4d1710c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:58.298151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370fe7d-15d3-454d-96ca-692fc4d1710c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:58.298462 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370fe7d-15d3-454d-96ca-692fc4d1710c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:58.298738 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370fe7d-15d3-454d-96ca-692fc4d1710c None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:58.299482 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370fe7d-15d3-454d-96ca-692fc4d1710c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:52:58.299786 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370fe7d-15d3-454d-96ca-692fc4d1710c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:58.301179 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370fe7d-15d3-454d-96ca-692fc4d1710c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:58.309747 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 602/1189] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:52:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:58.316478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f37e99-60d8-403b-89c0-9f98edf1d680 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:58.422338 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f37e99-60d8-403b-89c0-9f98edf1d680 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:58.456485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110325e-1165-4825-aab7-56cb1d830f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:58.457360 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110325e-1165-4825-aab7-56cb1d830f4e None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:58.457642 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110325e-1165-4825-aab7-56cb1d830f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:58.457918 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110325e-1165-4825-aab7-56cb1d830f4e None admin] PATH_INFO: `/v3/users/8bf41df333374fb5b08553bd6ba0143e` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:58.459149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110325e-1165-4825-aab7-56cb1d830f4e None admin] RBAC: Authorizing `identity:get_user(user_id=8bf41df333374fb5b08553bd6ba0143e)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:58.460588 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110325e-1165-4825-aab7-56cb1d830f4e None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:58.462249 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 588/1190] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:52:58 2022] GET /identity/v3/users/8bf41df333374fb5b08553bd6ba0143e => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:58.468083 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370fe7d-15d3-454d-96ca-692fc4d1710c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:58.515179 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:58.579902 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370fe7d-15d3-454d-96ca-692fc4d1710c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:58.613653 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c9968-8553-4278-9f16-bed0cec74967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:58.614548 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c9968-8553-4278-9f16-bed0cec74967 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:58.614830 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c9968-8553-4278-9f16-bed0cec74967 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:58.615105 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c9968-8553-4278-9f16-bed0cec74967 None admin] PATH_INFO: `/v3/projects/1cb44a82ee6b4672a8c3dfffe6936b6d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:58.616288 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c9968-8553-4278-9f16-bed0cec74967 None admin] RBAC: Authorizing `identity:get_project(project_id=1cb44a82ee6b4672a8c3dfffe6936b6d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:58.617712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c9968-8553-4278-9f16-bed0cec74967 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:58.619460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 603/1191] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:52:58 2022] GET /identity/v3/projects/1cb44a82ee6b4672a8c3dfffe6936b6d => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:52:58.626200 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110325e-1165-4825-aab7-56cb1d830f4e None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:52:58.687080 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:52:58.740614 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110325e-1165-4825-aab7-56cb1d830f4e None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:52:58.776550 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855d2a4-2c44-403e-b3db-7879c85d6392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:52:58.777459 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855d2a4-2c44-403e-b3db-7879c85d6392 None admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:52:58.777741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855d2a4-2c44-403e-b3db-7879c85d6392 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:52:58.778025 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855d2a4-2c44-403e-b3db-7879c85d6392 None admin] PATH_INFO: `/v3/projects/1cb44a82ee6b4672a8c3dfffe6936b6d/users/8bf41df333374fb5b08553bd6ba0143e/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:52:58.780133 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855d2a4-2c44-403e-b3db-7879c85d6392 None admin] RBAC: Authorizing `identity:create_grant(project_id=1cb44a82ee6b4672a8c3dfffe6936b6d, user_id=8bf41df333374fb5b08553bd6ba0143e, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:52:58.781879 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855d2a4-2c44-403e-b3db-7879c85d6392 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:52:58.798910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 589/1192] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:52:58 2022] PUT /identity/v3/projects/1cb44a82ee6b4672a8c3dfffe6936b6d/users/8bf41df333374fb5b08553bd6ba0143e/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:53:01.036830 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb81a62-823e-4cb9-be8e-df7b3f4013c4 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:01.037376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb81a62-823e-4cb9-be8e-df7b3f4013c4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:01.037376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb81a62-823e-4cb9-be8e-df7b3f4013c4 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:01.038806 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 604/1193] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:53:01.044043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d7d41-67f1-4833-a3bd-e56c6056ec28 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:01.044459 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d7d41-67f1-4833-a3bd-e56c6056ec28 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:01.044600 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d7d41-67f1-4833-a3bd-e56c6056ec28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:01.061470 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-446d7d41-67f1-4833-a3bd-e56c6056ec28 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:53:01.062800 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-446d7d41-67f1-4833-a3bd-e56c6056ec28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:53:01.130122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 590/1194] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:53:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:53:01.154429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb81a62-823e-4cb9-be8e-df7b3f4013c4 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:53:01.264388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb81a62-823e-4cb9-be8e-df7b3f4013c4 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:53:01.280082 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:53:01.305417 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c340a-1f55-4a88-b397-6aab394ff8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:53:01.306139 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c340a-1f55-4a88-b397-6aab394ff8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:01.306449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c340a-1f55-4a88-b397-6aab394ff8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:01.306721 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c340a-1f55-4a88-b397-6aab394ff8c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:01.321313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c340a-1f55-4a88-b397-6aab394ff8c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:53:01.329588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-7e8c340a-1f55-4a88-b397-6aab394ff8c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:53:01.329588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:53:01.329588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:53:01.329588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:53:01.329588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:53:01.329588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:53:01.329588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:53:01.329588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:53:01.329588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:53:01.329588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:53:01.329588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:53:01.329588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:53:01.329588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:53:01.329588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:53:01.329588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:53:01.329588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:53:01.330839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:53:01.330839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:53:01.330839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:53:01.330839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:53:01.330839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:53:01.330839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:53:01.330839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:53:01.330839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:53:01.330839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:53:01.330839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:53:01.330839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:53:01.330839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:53:01.330839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:53:01.330839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:53:01.330839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:53:01.330839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:53:01.332095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:53:01.332095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:53:01.332095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:53:01.332095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:53:01.332095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:53:01.332095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:53:01.332095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:53:01.332095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:53:01.332095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:53:01.332095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:53:01.332095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:53:01.332095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:53:01.332095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 605/1195] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:53:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:53:01.337121 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d7d41-67f1-4833-a3bd-e56c6056ec28 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:53:01.377904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:53:01.439909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d7d41-67f1-4833-a3bd-e56c6056ec28 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:53:01.478202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c2b3d-35d7-4989-8389-5ede68dbb11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:53:01.478880 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c2b3d-35d7-4989-8389-5ede68dbb11c None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:01.479172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c2b3d-35d7-4989-8389-5ede68dbb11c None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:01.479429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c2b3d-35d7-4989-8389-5ede68dbb11c None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:01.480180 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c2b3d-35d7-4989-8389-5ede68dbb11c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:53:01.480484 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c2b3d-35d7-4989-8389-5ede68dbb11c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:53:01.481862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c2b3d-35d7-4989-8389-5ede68dbb11c None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:53:01.491862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 591/1196] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:53:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:53:01.498626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c340a-1f55-4a88-b397-6aab394ff8c2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:53:01.609328 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c340a-1f55-4a88-b397-6aab394ff8c2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:53:01.644518 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d7861-3eb0-4187-98b6-83e6445578b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:53:01.645395 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d7861-3eb0-4187-98b6-83e6445578b1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:01.645676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d7861-3eb0-4187-98b6-83e6445578b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:01.645954 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d7861-3eb0-4187-98b6-83e6445578b1 None admin] PATH_INFO: `/v3/users/8bf41df333374fb5b08553bd6ba0143e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:01.647202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d7861-3eb0-4187-98b6-83e6445578b1 None admin] RBAC: Authorizing `identity:get_user(user_id=8bf41df333374fb5b08553bd6ba0143e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:53:01.648677 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d7861-3eb0-4187-98b6-83e6445578b1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:53:01.650388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 606/1197] 10.222.0.76 () {56 vars in 1177 bytes} [Thu May 26 17:53:01 2022] GET /identity/v3/users/8bf41df333374fb5b08553bd6ba0143e => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:53:01.656346 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c2b3d-35d7-4989-8389-5ede68dbb11c None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:53:01.761633 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:53:01.765016 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c2b3d-35d7-4989-8389-5ede68dbb11c None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:53:01.803714 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbfb7a-6eb2-40e2-ae1d-eac02866d703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:53:01.804369 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbfb7a-6eb2-40e2-ae1d-eac02866d703 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:01.804651 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbfb7a-6eb2-40e2-ae1d-eac02866d703 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:01.804931 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbfb7a-6eb2-40e2-ae1d-eac02866d703 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:01.875832 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfbfb7a-6eb2-40e2-ae1d-eac02866d703 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:53:01.877288 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfbfb7a-6eb2-40e2-ae1d-eac02866d703 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:53:01.975200 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:53:01.982322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 592/1198] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:53:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:53:01.985160 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d7861-3eb0-4187-98b6-83e6445578b1 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:53:02.027353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:53:02.086953 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d7861-3eb0-4187-98b6-83e6445578b1 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:53:02.119464 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80fc30-9846-40f0-b737-5200df4cc020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:53:02.120319 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80fc30-9846-40f0-b737-5200df4cc020 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:02.120603 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80fc30-9846-40f0-b737-5200df4cc020 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:02.120879 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80fc30-9846-40f0-b737-5200df4cc020 None admin] PATH_INFO: `/v3/projects/1cb44a82ee6b4672a8c3dfffe6936b6d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:02.122032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80fc30-9846-40f0-b737-5200df4cc020 None admin] RBAC: Authorizing `identity:get_project(project_id=1cb44a82ee6b4672a8c3dfffe6936b6d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:53:02.123485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80fc30-9846-40f0-b737-5200df4cc020 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:53:02.125974 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 607/1199] 10.222.0.76 () {56 vars in 1186 bytes} [Thu May 26 17:53:01 2022] GET /identity/v3/projects/1cb44a82ee6b4672a8c3dfffe6936b6d => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:53:02.132295 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbfb7a-6eb2-40e2-ae1d-eac02866d703 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:53:02.233484 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbfb7a-6eb2-40e2-ae1d-eac02866d703 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:53:02.269458 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1606e4-15b8-4890-a06e-522bfdb6c7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:53:02.270114 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1606e4-15b8-4890-a06e-522bfdb6c7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:53:02.270425 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1606e4-15b8-4890-a06e-522bfdb6c7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:53:02.270700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1606e4-15b8-4890-a06e-522bfdb6c7e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:53:02.271522 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1606e4-15b8-4890-a06e-522bfdb6c7e3 None admin] RBAC: Adding query filter params (role.id=4fa8f06085c741a5b3cf64e63cc0754f, scope.project.id=1cb44a82ee6b4672a8c3dfffe6936b6d, user.id=8bf41df333374fb5b08553bd6ba0143e) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:53:02.271831 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1606e4-15b8-4890-a06e-522bfdb6c7e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:53:02.273248 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1606e4-15b8-4890-a06e-522bfdb6c7e3 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:53:02.279153 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 593/1200] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:53:02 2022] GET /identity/v3/role_assignments?user.id=8bf41df333374fb5b08553bd6ba0143e&scope.project.id=1cb44a82ee6b4672a8c3dfffe6936b6d&role.id=4fa8f06085c741a5b3cf64e63cc0754f => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:53:12.115659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=46 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:53:12.276523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=68 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:06.588678 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd16ec7-9904-4c35-ab4d-a4b5d88ed320 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:06.589342 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd16ec7-9904-4c35-ab4d-a4b5d88ed320 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:06.589342 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd16ec7-9904-4c35-ab4d-a4b5d88ed320 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:06.590870 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 608/1201] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:54:06.598767 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926e608-42b4-45f0-8d4f-ea82ba074b94 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:06.598767 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926e608-42b4-45f0-8d4f-ea82ba074b94 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:06.598767 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926e608-42b4-45f0-8d4f-ea82ba074b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:06.622622 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-d926e608-42b4-45f0-8d4f-ea82ba074b94 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:54:06.625512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-d926e608-42b4-45f0-8d4f-ea82ba074b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:54:06.736624 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 594/1202] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:54:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:54:06.764850 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd16ec7-9904-4c35-ab4d-a4b5d88ed320 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:06.909054 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd16ec7-9904-4c35-ab4d-a4b5d88ed320 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:06.959662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb224a9-9fc8-49be-aca0-301f71167d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:06.961707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb224a9-9fc8-49be-aca0-301f71167d24 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:06.962580 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb224a9-9fc8-49be-aca0-301f71167d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:06.963605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb224a9-9fc8-49be-aca0-301f71167d24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:06.987546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb224a9-9fc8-49be-aca0-301f71167d24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:06.991288 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb224a9-9fc8-49be-aca0-301f71167d24 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:07.016388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-afb224a9-9fc8-49be-aca0-301f71167d24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:07.016388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:07.016388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:07.016388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:07.016388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:07.016388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:07.016388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:07.016388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:07.016388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:07.016388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:07.016388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:07.016388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:07.016388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:54:07.016388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:54:07.016388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:54:07.018522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:54:07.018522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:07.018522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:07.018522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:07.018522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:07.018522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:07.018522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:07.018522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:07.018522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:54:07.018522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:07.018522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:07.018522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:07.018522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:07.018522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:07.018522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:07.018522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:07.018522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:07.020494 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:54:07.020494 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:54:07.020494 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:07.020494 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:54:07.023335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 609/1203] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:54:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:54:07.024379 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926e608-42b4-45f0-8d4f-ea82ba074b94 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:07.165176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926e608-42b4-45f0-8d4f-ea82ba074b94 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:07.199005 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:07.205622 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2b327-e9fc-4707-b89e-2bce49d0b2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:07.206330 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2b327-e9fc-4707-b89e-2bce49d0b2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:07.206619 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2b327-e9fc-4707-b89e-2bce49d0b2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:07.206897 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2b327-e9fc-4707-b89e-2bce49d0b2e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:07.207639 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d2b327-e9fc-4707-b89e-2bce49d0b2e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:07.207939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d2b327-e9fc-4707-b89e-2bce49d0b2e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:07.209343 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d2b327-e9fc-4707-b89e-2bce49d0b2e1 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:07.222473 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 595/1204] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:07.229503 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb224a9-9fc8-49be-aca0-301f71167d24 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:07.296400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:07.353407 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb224a9-9fc8-49be-aca0-301f71167d24 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:07.393682 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89a2ec-d2aa-4d0a-a76f-aba24f47026c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:07.394430 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89a2ec-d2aa-4d0a-a76f-aba24f47026c None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:07.394738 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89a2ec-d2aa-4d0a-a76f-aba24f47026c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:07.395040 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89a2ec-d2aa-4d0a-a76f-aba24f47026c None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:07.395777 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89a2ec-d2aa-4d0a-a76f-aba24f47026c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:07.397324 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89a2ec-d2aa-4d0a-a76f-aba24f47026c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:07.423402 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 610/1205] 10.222.0.76 () {60 vars in 1132 bytes} [Thu May 26 17:54:07 2022] POST /identity/v3/users => generated 251 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:54:09.991099 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3164cb-c2c4-403c-8e52-9281ba359242 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:09.991928 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3164cb-c2c4-403c-8e52-9281ba359242 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:09.991928 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3164cb-c2c4-403c-8e52-9281ba359242 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:09.993404 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 596/1206] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:54:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:54:09.999322 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c8006-239e-4cb7-9720-4fa997c83741 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:09.999842 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c8006-239e-4cb7-9720-4fa997c83741 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:10.000074 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c8006-239e-4cb7-9720-4fa997c83741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:10.018991 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a43c8006-239e-4cb7-9720-4fa997c83741 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:54:10.020361 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a43c8006-239e-4cb7-9720-4fa997c83741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:54:10.100228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 611/1207] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:54:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:54:10.124221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3164cb-c2c4-403c-8e52-9281ba359242 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:10.245864 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3164cb-c2c4-403c-8e52-9281ba359242 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:10.287992 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b96a9-c9c9-49c5-80df-46f552d9e537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:10.289122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b96a9-c9c9-49c5-80df-46f552d9e537 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:10.289420 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b96a9-c9c9-49c5-80df-46f552d9e537 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:10.289711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b96a9-c9c9-49c5-80df-46f552d9e537 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:10.303862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b96a9-c9c9-49c5-80df-46f552d9e537 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:54:10.312565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-e48b96a9-c9c9-49c5-80df-46f552d9e537 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:54:10.312565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:10.312565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:10.312565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:10.312565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:10.312565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:10.312565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:10.312565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:10.312565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:10.312565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:10.312565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:10.312565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:10.312565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:54:10.312565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:54:10.312565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:54:10.312565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:54:10.313811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:54:10.313811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:54:10.313811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:54:10.313811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:54:10.313811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:10.313811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:10.313811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:10.313811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:10.313811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:10.313811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:10.313811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:10.313811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:54:10.313811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:10.313811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:10.313811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:10.313811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:10.314938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:10.314938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:10.314938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:10.314938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:10.314938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:54:10.314938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:54:10.314938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:54:10.314938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:54:10.314938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:54:10.314938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:54:10.314938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:54:10.314938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:10.314938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 597/1208] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:54:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:10.319949 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c8006-239e-4cb7-9720-4fa997c83741 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:10.369869 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:10.439651 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c8006-239e-4cb7-9720-4fa997c83741 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:10.480727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4a27d-536f-430c-9628-5bb79c532de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:10.481404 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4a27d-536f-430c-9628-5bb79c532de0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:10.481685 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4a27d-536f-430c-9628-5bb79c532de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:10.481958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4a27d-536f-430c-9628-5bb79c532de0 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:10.482738 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4a27d-536f-430c-9628-5bb79c532de0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:10.483150 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4a27d-536f-430c-9628-5bb79c532de0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:10.485164 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4a27d-536f-430c-9628-5bb79c532de0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:10.494106 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 612/1209] 10.222.0.76 () {56 vars in 1103 bytes} [Thu May 26 17:54:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:54:10.500978 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b96a9-c9c9-49c5-80df-46f552d9e537 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:10.571646 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:10.653408 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b96a9-c9c9-49c5-80df-46f552d9e537 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:10.703012 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a510641-6012-472c-ba30-f255cd5a662a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:10.703575 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a510641-6012-472c-ba30-f255cd5a662a None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:10.703865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a510641-6012-472c-ba30-f255cd5a662a None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:10.704147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a510641-6012-472c-ba30-f255cd5a662a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:10.721663 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a510641-6012-472c-ba30-f255cd5a662a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:10.723270 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a510641-6012-472c-ba30-f255cd5a662a None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:10.743284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-8a510641-6012-472c-ba30-f255cd5a662a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:10.743284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:10.743284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:10.743284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:10.743284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:10.743284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:10.743284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:10.743284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:10.743284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:10.743284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:10.743284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:10.743284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:10.743284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:54:10.743284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:54:10.743284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:54:10.744325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:54:10.744325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:10.744325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:10.744325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:10.744325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:10.744325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:10.744325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:10.744325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:10.744325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:54:10.744325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:10.744325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:10.744325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:10.744325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:10.744325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:10.744325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:10.744325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:10.744325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:10.745259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:54:10.745259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:54:10.745259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:10.745259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:10.745259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 598/1210] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:54:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:54:10.752383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4a27d-536f-430c-9628-5bb79c532de0 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:10.888316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:10.898826 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4a27d-536f-430c-9628-5bb79c532de0 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:10.942258 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf082890-ffcf-452d-b36d-8ba679e462d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:10.942908 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf082890-ffcf-452d-b36d-8ba679e462d0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:10.943189 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf082890-ffcf-452d-b36d-8ba679e462d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:10.943464 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf082890-ffcf-452d-b36d-8ba679e462d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:10.944194 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf082890-ffcf-452d-b36d-8ba679e462d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:10.944555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf082890-ffcf-452d-b36d-8ba679e462d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:10.945969 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf082890-ffcf-452d-b36d-8ba679e462d0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:10.960110 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 613/1211] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:10.965561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a510641-6012-472c-ba30-f255cd5a662a None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:11.101539 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a510641-6012-472c-ba30-f255cd5a662a None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:11.102705 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:11.152903 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33c820-952e-403c-8265-f2bf13b72b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:11.153754 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33c820-952e-403c-8265-f2bf13b72b24 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:11.154012 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33c820-952e-403c-8265-f2bf13b72b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:11.154327 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33c820-952e-403c-8265-f2bf13b72b24 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:11.163856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d33c820-952e-403c-8265-f2bf13b72b24 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:11.165353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d33c820-952e-403c-8265-f2bf13b72b24 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:11.183996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-5d33c820-952e-403c-8265-f2bf13b72b24 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 17:54:11.183996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:11.183996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:11.183996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:11.183996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:11.183996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:11.183996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:11.183996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:11.183996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:11.183996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:11.183996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:11.183996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:11.183996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:54:11.183996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:54:11.183996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:54:11.183996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:54:11.185095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:11.185095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:11.185095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:54:11.185095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:54:11.185095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:54:11.185095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:54:11.185095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:11.185095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:11.185095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:11.185095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:11.185095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:11.185095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:54:11.185095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:11.185095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:11.185095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:11.185095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:11.185095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:11.185095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:11.186229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:11.186229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:11.186229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:54:11.186229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:54:11.186229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:54:11.186229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:54:11.186229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:54:11.186229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:54:11.186229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 17:54:11.186229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:11.186229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 599/1212] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:54:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:11.191208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf082890-ffcf-452d-b36d-8ba679e462d0 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:11.323894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf082890-ffcf-452d-b36d-8ba679e462d0 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:11.362934 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44d805-75b9-489e-9f76-156b4fb9d873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:11.363531 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44d805-75b9-489e-9f76-156b4fb9d873 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:11.363817 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44d805-75b9-489e-9f76-156b4fb9d873 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:11.364094 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44d805-75b9-489e-9f76-156b4fb9d873 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:11.365109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a44d805-75b9-489e-9f76-156b4fb9d873 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:11.365427 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a44d805-75b9-489e-9f76-156b4fb9d873 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:11.367939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a44d805-75b9-489e-9f76-156b4fb9d873 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:11.393220 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 614/1213] 10.222.0.76 () {56 vars in 1143 bytes} [Thu May 26 17:54:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:11.394783 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33c820-952e-403c-8265-f2bf13b72b24 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:11.550657 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33c820-952e-403c-8265-f2bf13b72b24 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:11.596242 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97618b-c07f-453e-b336-3973dbe15818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:11.597031 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97618b-c07f-453e-b336-3973dbe15818 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:11.597222 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97618b-c07f-453e-b336-3973dbe15818 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:11.597485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97618b-c07f-453e-b336-3973dbe15818 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:11.634536 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:11.687275 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97618b-c07f-453e-b336-3973dbe15818 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:11.688777 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97618b-c07f-453e-b336-3973dbe15818 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:11.844480 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 600/1214] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:54:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:54:11.851569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44d805-75b9-489e-9f76-156b4fb9d873 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:11.875317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:11.977964 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44d805-75b9-489e-9f76-156b4fb9d873 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:12.023218 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819af5c-2fbe-4400-aebd-1a57ee6764b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:12.023987 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819af5c-2fbe-4400-aebd-1a57ee6764b3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:12.024328 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819af5c-2fbe-4400-aebd-1a57ee6764b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:12.024695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819af5c-2fbe-4400-aebd-1a57ee6764b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:12.037634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819af5c-2fbe-4400-aebd-1a57ee6764b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:12.039183 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819af5c-2fbe-4400-aebd-1a57ee6764b3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:12.060012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-a819af5c-2fbe-4400-aebd-1a57ee6764b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:12.060012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:12.060012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:12.060012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:12.060012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:12.060012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:12.060012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:12.060012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:12.060012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:12.060012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:12.060012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:12.060012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:12.060012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:54:12.060012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:54:12.060012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:54:12.061922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:54:12.061922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:12.061922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:12.061922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:12.061922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:12.061922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:12.061922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:12.061922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:12.061922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:54:12.061922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:12.061922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:12.061922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:12.061922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:12.061922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:12.061922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:12.061922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:12.061922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:12.062896 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:54:12.062896 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:54:12.062896 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:12.062896 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:54:12.062896 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 615/1215] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:12.067990 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97618b-c07f-453e-b336-3973dbe15818 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:12.147487 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:12.198713 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97618b-c07f-453e-b336-3973dbe15818 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:12.244106 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4961af-feaf-47e0-8047-61e8590de21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:12.244945 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4961af-feaf-47e0-8047-61e8590de21b None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:12.245246 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4961af-feaf-47e0-8047-61e8590de21b None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:12.245594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4961af-feaf-47e0-8047-61e8590de21b None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:12.246301 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4961af-feaf-47e0-8047-61e8590de21b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:12.246652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4961af-feaf-47e0-8047-61e8590de21b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:12.248035 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4961af-feaf-47e0-8047-61e8590de21b None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:12.262413 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 601/1216] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:12.269817 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819af5c-2fbe-4400-aebd-1a57ee6764b3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:12.346058 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:12.381234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819af5c-2fbe-4400-aebd-1a57ee6764b3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:12.419493 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865f288-6478-47b6-a90e-20590a4b49ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:12.421177 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865f288-6478-47b6-a90e-20590a4b49ce None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:12.421177 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865f288-6478-47b6-a90e-20590a4b49ce None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:12.421613 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865f288-6478-47b6-a90e-20590a4b49ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:12.429272 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865f288-6478-47b6-a90e-20590a4b49ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:12.430854 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865f288-6478-47b6-a90e-20590a4b49ce None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:12.447194 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-e865f288-6478-47b6-a90e-20590a4b49ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:54:12.447194 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:12.447194 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:12.447194 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:12.447194 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:12.447194 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:12.447194 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:12.447194 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:12.447194 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:12.447194 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:12.447194 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:12.447194 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:12.447194 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:54:12.447194 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:54:12.447194 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:54:12.449851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:54:12.449851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:12.449851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:12.449851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:12.449851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:12.449851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:12.449851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:12.449851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:12.449851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:54:12.449851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:12.449851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:12.449851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:12.449851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:12.449851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:12.449851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:12.449851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:12.449851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:12.451185 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:54:12.451185 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:54:12.451185 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:54:12.451185 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:54:12.451185 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:54:12.451185 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:54:12.451185 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:54:12.451185 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:54:12.451185 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 616/1217] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:54:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:12.456482 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4961af-feaf-47e0-8047-61e8590de21b None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:12.585713 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4961af-feaf-47e0-8047-61e8590de21b None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:12.611855 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:12.624938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36fdc6-6fb7-4e27-a863-98cdad478d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:12.625611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36fdc6-6fb7-4e27-a863-98cdad478d81 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:12.625896 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36fdc6-6fb7-4e27-a863-98cdad478d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:12.626171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36fdc6-6fb7-4e27-a863-98cdad478d81 None admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:12.627000 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36fdc6-6fb7-4e27-a863-98cdad478d81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:12.627318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36fdc6-6fb7-4e27-a863-98cdad478d81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:12.629011 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36fdc6-6fb7-4e27-a863-98cdad478d81 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:12.643633 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 602/1218] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:54:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:54:12.650455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865f288-6478-47b6-a90e-20590a4b49ce None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:12.781742 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865f288-6478-47b6-a90e-20590a4b49ce None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:12.838134 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:12.849565 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae107a-94e5-4833-a799-6539f43eca81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:12.850635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae107a-94e5-4833-a799-6539f43eca81 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:12.851111 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae107a-94e5-4833-a799-6539f43eca81 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:12.851270 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae107a-94e5-4833-a799-6539f43eca81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:12.852433 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae107a-94e5-4833-a799-6539f43eca81 None admin] RBAC: Adding query filter params (role.id=e776303775a2423aa0c62c161ecb5a7e, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=596f126382e3440fb3cf7c357eef889f) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:12.852934 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae107a-94e5-4833-a799-6539f43eca81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:12.854690 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae107a-94e5-4833-a799-6539f43eca81 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:12.868499 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 617/1219] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:54:12 2022] GET /identity/v3/role_assignments?user.id=596f126382e3440fb3cf7c357eef889f&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=e776303775a2423aa0c62c161ecb5a7e => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:15.207914 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a850d0-ad05-4c0c-95a0-6480527906fb None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:15.208537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a850d0-ad05-4c0c-95a0-6480527906fb None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:15.208537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a850d0-ad05-4c0c-95a0-6480527906fb None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:15.209838 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 603/1220] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:54:15.216534 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea628d5-9521-4c41-81cb-c8116b02df10 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:15.217871 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea628d5-9521-4c41-81cb-c8116b02df10 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:15.218113 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea628d5-9521-4c41-81cb-c8116b02df10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:15.241743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-dea628d5-9521-4c41-81cb-c8116b02df10 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:54:15.243044 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-dea628d5-9521-4c41-81cb-c8116b02df10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:54:15.334104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 618/1221] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:54:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:54:15.357360 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a850d0-ad05-4c0c-95a0-6480527906fb None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:15.490938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a850d0-ad05-4c0c-95a0-6480527906fb None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:15.537623 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f0aba-be73-4940-92f7-ca2e3f350def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:15.538473 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f0aba-be73-4940-92f7-ca2e3f350def None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:15.538759 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f0aba-be73-4940-92f7-ca2e3f350def None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:15.539030 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f0aba-be73-4940-92f7-ca2e3f350def None admin] PATH_INFO: `/v3/roles/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:15.550844 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f0aba-be73-4940-92f7-ca2e3f350def None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:54:15.558652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-5f2f0aba-be73-4940-92f7-ca2e3f350def None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:54:15.558652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:15.558652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:15.558652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:15.558652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:15.558652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:15.558652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:15.558652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:15.558652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:15.558652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:15.558652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:15.558652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:15.558652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:54:15.558652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:54:15.558652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:54:15.558652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:54:15.559761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:54:15.559761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:54:15.559761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:54:15.559761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:54:15.559761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:15.559761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:15.559761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:15.559761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:15.559761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:15.559761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:15.559761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:15.559761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:54:15.559761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:15.559761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:15.559761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:15.559761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:15.560649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:15.560649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:15.560649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:15.560649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:15.560649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:54:15.560649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:54:15.560649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:54:15.560649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:54:15.560649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:54:15.560649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:54:15.560649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:54:15.560649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:15.568439 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 604/1222] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:54:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:15.568605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea628d5-9521-4c41-81cb-c8116b02df10 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:15.674663 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea628d5-9521-4c41-81cb-c8116b02df10 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:15.699859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:15.711701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703402f6-451c-429e-a790-1c00d9425fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:15.712466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703402f6-451c-429e-a790-1c00d9425fdd None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:15.712783 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703402f6-451c-429e-a790-1c00d9425fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:15.713033 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703402f6-451c-429e-a790-1c00d9425fdd None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:15.713825 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703402f6-451c-429e-a790-1c00d9425fdd None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:15.714135 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703402f6-451c-429e-a790-1c00d9425fdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:15.715550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703402f6-451c-429e-a790-1c00d9425fdd None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:15.728926 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 619/1223] 10.222.0.76 () {56 vars in 1103 bytes} [Thu May 26 17:54:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:15.730125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f0aba-be73-4940-92f7-ca2e3f350def None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:15.781933 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:15.855061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f0aba-be73-4940-92f7-ca2e3f350def None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:15.899858 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5dd518-3df8-4d8d-9484-287d28b2275a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:15.900659 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5dd518-3df8-4d8d-9484-287d28b2275a None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:15.900951 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5dd518-3df8-4d8d-9484-287d28b2275a None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:15.901218 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5dd518-3df8-4d8d-9484-287d28b2275a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:15.913238 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5dd518-3df8-4d8d-9484-287d28b2275a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:15.914741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5dd518-3df8-4d8d-9484-287d28b2275a None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:15.933652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-3e5dd518-3df8-4d8d-9484-287d28b2275a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:15.933652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:15.933652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:15.933652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:15.933652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:15.933652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:15.933652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:15.933652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:15.933652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:15.933652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:15.933652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:15.933652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:15.933652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:54:15.933652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:54:15.933652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:54:15.934900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:54:15.934900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:15.934900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:15.934900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:15.934900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:15.934900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:15.934900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:15.934900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:15.934900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:54:15.934900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:15.934900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:15.934900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:15.934900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:15.934900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:15.934900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:15.934900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:15.934900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:15.941625 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:54:15.941625 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:54:15.941625 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:15.941625 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:15.941625 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 605/1224] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:15.941993 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703402f6-451c-429e-a790-1c00d9425fdd None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:16.057091 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703402f6-451c-429e-a790-1c00d9425fdd None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:16.102427 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc143c29-169a-426f-a3cc-f9c5d36916f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:16.103532 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc143c29-169a-426f-a3cc-f9c5d36916f2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:16.104286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc143c29-169a-426f-a3cc-f9c5d36916f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:16.104680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc143c29-169a-426f-a3cc-f9c5d36916f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:16.105848 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc143c29-169a-426f-a3cc-f9c5d36916f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:16.106422 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc143c29-169a-426f-a3cc-f9c5d36916f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:16.108567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc143c29-169a-426f-a3cc-f9c5d36916f2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:16.122365 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 620/1225] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:16.131198 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5dd518-3df8-4d8d-9484-287d28b2275a None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:16.231836 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:16.240021 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5dd518-3df8-4d8d-9484-287d28b2275a None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:16.275536 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5540e-8966-4adb-952a-799e37c253ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:16.276347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5540e-8966-4adb-952a-799e37c253ad None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:16.276632 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5540e-8966-4adb-952a-799e37c253ad None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:16.276912 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5540e-8966-4adb-952a-799e37c253ad None admin] PATH_INFO: `/v3/users/placement` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:16.285854 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5540e-8966-4adb-952a-799e37c253ad None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:16.287329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5540e-8966-4adb-952a-799e37c253ad None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:16.304656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-2bf5540e-8966-4adb-952a-799e37c253ad None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 17:54:16.304656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:16.304656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:16.304656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:16.304656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:16.304656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:16.304656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:16.304656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:16.304656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:16.304656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:16.304656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:16.304656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:16.304656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:54:16.304656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:54:16.304656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:54:16.304656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:54:16.306078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:16.306078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:16.306078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:54:16.306078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:54:16.306078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:54:16.306078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:54:16.306078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:16.306078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:16.306078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:16.306078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:16.306078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:16.306078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:54:16.306078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:16.306078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:16.306078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:16.306078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:16.306078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:16.306078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:16.307600 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:16.307600 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:16.307600 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:54:16.307600 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:54:16.307600 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:54:16.307600 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:54:16.307600 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:54:16.307600 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:54:16.307600 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 17:54:16.307600 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:16.307600 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 606/1226] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:54:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:16.315032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc143c29-169a-426f-a3cc-f9c5d36916f2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:16.368107 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:16.427539 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc143c29-169a-426f-a3cc-f9c5d36916f2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:16.467765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000e380-083f-411f-8e5c-08cb4fae198a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:16.468444 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000e380-083f-411f-8e5c-08cb4fae198a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:16.468734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000e380-083f-411f-8e5c-08cb4fae198a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:16.469016 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000e380-083f-411f-8e5c-08cb4fae198a None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:16.469861 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e000e380-083f-411f-8e5c-08cb4fae198a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:16.470171 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e000e380-083f-411f-8e5c-08cb4fae198a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:16.471647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e000e380-083f-411f-8e5c-08cb4fae198a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:16.494298 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 621/1227] 10.222.0.76 () {56 vars in 1143 bytes} [Thu May 26 17:54:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:16.500519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5540e-8966-4adb-952a-799e37c253ad None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:16.622689 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5540e-8966-4adb-952a-799e37c253ad None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:16.662115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59702a08-2c56-4362-b6fa-940b46a4dc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:16.663038 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59702a08-2c56-4362-b6fa-940b46a4dc52 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:16.663314 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59702a08-2c56-4362-b6fa-940b46a4dc52 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:16.663586 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59702a08-2c56-4362-b6fa-940b46a4dc52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:16.675686 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59702a08-2c56-4362-b6fa-940b46a4dc52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:16.677202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59702a08-2c56-4362-b6fa-940b46a4dc52 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:16.697621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-59702a08-2c56-4362-b6fa-940b46a4dc52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:16.697621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:16.697621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:16.697621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:16.697621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:16.697621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:16.697621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:16.697621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:16.697621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:16.697621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:16.697621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:16.697621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:16.697621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:54:16.697621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:54:16.697621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:54:16.698814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:54:16.698814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:16.698814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:16.698814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:16.698814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:16.698814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:16.698814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:16.698814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:16.698814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:54:16.698814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:16.698814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:16.698814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:16.698814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:16.698814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:16.698814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:16.698814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:16.698814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:16.699965 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:54:16.699965 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:54:16.699965 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:16.699965 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:16.699965 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 607/1228] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:54:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:16.705164 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000e380-083f-411f-8e5c-08cb4fae198a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:16.830679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000e380-083f-411f-8e5c-08cb4fae198a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:16.833271 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:16.873273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e505f48-01fe-492c-9937-d1854428b0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:16.876059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e505f48-01fe-492c-9937-d1854428b0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:16.876059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e505f48-01fe-492c-9937-d1854428b0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:16.876059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e505f48-01fe-492c-9937-d1854428b0c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:16.876059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e505f48-01fe-492c-9937-d1854428b0c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:16.876436 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e505f48-01fe-492c-9937-d1854428b0c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:16.877554 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e505f48-01fe-492c-9937-d1854428b0c1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:16.890213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 622/1229] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:16.896191 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59702a08-2c56-4362-b6fa-940b46a4dc52 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:16.910582 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:17.022880 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59702a08-2c56-4362-b6fa-940b46a4dc52 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:17.069091 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510fc06-13a7-44c8-be57-99fb029e6f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:17.069909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510fc06-13a7-44c8-be57-99fb029e6f01 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:17.070215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510fc06-13a7-44c8-be57-99fb029e6f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:17.070510 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510fc06-13a7-44c8-be57-99fb029e6f01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:17.079121 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1510fc06-13a7-44c8-be57-99fb029e6f01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:17.080803 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1510fc06-13a7-44c8-be57-99fb029e6f01 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:17.097885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-1510fc06-13a7-44c8-be57-99fb029e6f01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:54:17.097885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:17.097885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:17.097885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:17.097885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:17.097885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:17.097885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:17.097885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:17.097885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:17.097885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:17.097885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:17.097885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:17.097885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:54:17.097885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:54:17.097885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:54:17.099925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:54:17.099925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:17.099925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:17.099925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:17.099925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:17.099925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:17.099925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:17.099925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:17.099925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:54:17.099925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:17.099925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:17.099925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:17.099925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:17.099925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:17.099925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:17.099925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:17.099925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:17.101949 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:54:17.101949 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:54:17.101949 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:54:17.101949 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:54:17.101949 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:54:17.101949 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:54:17.101949 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:54:17.101949 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:17.101949 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 608/1230] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:54:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:17.106505 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e505f48-01fe-492c-9937-d1854428b0c1 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:17.231596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e505f48-01fe-492c-9937-d1854428b0c1 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:17.271295 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0dd60-a291-4c5e-8ec6-e37797cd13ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:17.271977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0dd60-a291-4c5e-8ec6-e37797cd13ff None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:17.272293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0dd60-a291-4c5e-8ec6-e37797cd13ff None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:17.272538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0dd60-a291-4c5e-8ec6-e37797cd13ff None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:17.273326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f0dd60-a291-4c5e-8ec6-e37797cd13ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:17.273641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f0dd60-a291-4c5e-8ec6-e37797cd13ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:17.275115 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f0dd60-a291-4c5e-8ec6-e37797cd13ff None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:17.290019 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 623/1231] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:54:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:17.298292 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510fc06-13a7-44c8-be57-99fb029e6f01 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:17.374694 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:17.416228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510fc06-13a7-44c8-be57-99fb029e6f01 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:17.452067 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7ead9-0f8a-4ca8-9da8-f3c73a727717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:17.453841 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7ead9-0f8a-4ca8-9da8-f3c73a727717 None admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:17.454275 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7ead9-0f8a-4ca8-9da8-f3c73a727717 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:17.454680 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7ead9-0f8a-4ca8-9da8-f3c73a727717 None admin] PATH_INFO: `/v3/projects/ca1d09f491d944f99425a641561b03c3/users/596f126382e3440fb3cf7c357eef889f/roles/e776303775a2423aa0c62c161ecb5a7e` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:17.471939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7ead9-0f8a-4ca8-9da8-f3c73a727717 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1d09f491d944f99425a641561b03c3, user_id=596f126382e3440fb3cf7c357eef889f, role_id=e776303775a2423aa0c62c161ecb5a7e)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:17.473451 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7ead9-0f8a-4ca8-9da8-f3c73a727717 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:17.487221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 609/1232] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:54:17 2022] PUT /identity/v3/projects/ca1d09f491d944f99425a641561b03c3/users/596f126382e3440fb3cf7c357eef889f/roles/e776303775a2423aa0c62c161ecb5a7e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 17:54:19.866877 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354e82e-6619-4d67-8e59-991e91d0b7aa None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:19.867487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354e82e-6619-4d67-8e59-991e91d0b7aa None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:19.867487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354e82e-6619-4d67-8e59-991e91d0b7aa None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:19.869268 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 624/1233] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:54:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:54:19.876598 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc7d0d-ab88-4adf-a4ef-3dd015569541 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:19.877076 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc7d0d-ab88-4adf-a4ef-3dd015569541 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:19.877177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc7d0d-ab88-4adf-a4ef-3dd015569541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:19.896036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-2ecc7d0d-ab88-4adf-a4ef-3dd015569541 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:54:19.897345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-2ecc7d0d-ab88-4adf-a4ef-3dd015569541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:54:19.971867 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 610/1234] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:54:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:54:19.996220 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354e82e-6619-4d67-8e59-991e91d0b7aa None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:20.023112 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:20.115704 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354e82e-6619-4d67-8e59-991e91d0b7aa None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:20.157326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7923c37-0ec1-4de9-abe6-d07d32bf8335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:20.158271 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7923c37-0ec1-4de9-abe6-d07d32bf8335 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:20.158559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7923c37-0ec1-4de9-abe6-d07d32bf8335 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:20.158836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7923c37-0ec1-4de9-abe6-d07d32bf8335 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:20.171970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7923c37-0ec1-4de9-abe6-d07d32bf8335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:54:20.181810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-a7923c37-0ec1-4de9-abe6-d07d32bf8335 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 17:54:20.181810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:20.181810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:20.181810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:20.181810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:20.181810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:20.181810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:20.181810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:20.181810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:20.181810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:20.181810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:20.181810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:20.181810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:54:20.181810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:54:20.181810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:54:20.181810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:54:20.183389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:54:20.183389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:54:20.183389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:54:20.183389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:54:20.183389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:20.183389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:20.183389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:20.183389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:20.183389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:20.183389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:20.183389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:20.183389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:54:20.183389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:20.183389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:20.183389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:20.183389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:20.184293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:20.184293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:20.184293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:20.184293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:20.184293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:54:20.184293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:54:20.184293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:54:20.184293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:54:20.184293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:54:20.184293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:54:20.184293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 17:54:20.184293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:54:20.186750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 625/1235] 10.222.0.76 () {56 vars in 1102 bytes} [Thu May 26 17:54:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:20.190568 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc7d0d-ab88-4adf-a4ef-3dd015569541 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:20.244484 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:20.309923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc7d0d-ab88-4adf-a4ef-3dd015569541 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:20.354402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79c8cf-4a55-4569-96b9-67b0a492d001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:20.355081 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79c8cf-4a55-4569-96b9-67b0a492d001 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:20.355367 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79c8cf-4a55-4569-96b9-67b0a492d001 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:20.355704 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79c8cf-4a55-4569-96b9-67b0a492d001 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:20.356636 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b79c8cf-4a55-4569-96b9-67b0a492d001 None admin] RBAC: Adding query filter params (name=service) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:20.357002 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b79c8cf-4a55-4569-96b9-67b0a492d001 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:20.358755 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b79c8cf-4a55-4569-96b9-67b0a492d001 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:20.368796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 611/1236] 10.222.0.76 () {56 vars in 1103 bytes} [Thu May 26 17:54:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:54:20.374687 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7923c37-0ec1-4de9-abe6-d07d32bf8335 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:20.474069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:20.493245 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7923c37-0ec1-4de9-abe6-d07d32bf8335 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:20.533527 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe6e8e-3563-4cf1-ac61-6a17fa601404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:20.534744 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe6e8e-3563-4cf1-ac61-6a17fa601404 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:20.534744 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe6e8e-3563-4cf1-ac61-6a17fa601404 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:20.534962 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe6e8e-3563-4cf1-ac61-6a17fa601404 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:20.548954 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe6e8e-3563-4cf1-ac61-6a17fa601404 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:20.550450 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe6e8e-3563-4cf1-ac61-6a17fa601404 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:20.568280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-73fe6e8e-3563-4cf1-ac61-6a17fa601404 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:20.568280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:20.568280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:20.568280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:20.568280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:20.568280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:20.568280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:20.568280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:20.568280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:20.568280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:20.568280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:20.568280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:20.568280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:54:20.568280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:54:20.568280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:54:20.569400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:54:20.569400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:20.569400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:20.569400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:20.569400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:20.569400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:20.569400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:20.569400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:20.569400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:54:20.569400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:20.569400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:20.569400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:20.569400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:20.569400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:20.569400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:20.569400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:20.569400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:20.575847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:54:20.575847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:54:20.575847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:20.575847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:54:20.575847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 626/1237] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:54:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:20.576206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79c8cf-4a55-4569-96b9-67b0a492d001 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:20.693713 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79c8cf-4a55-4569-96b9-67b0a492d001 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:20.704686 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:20.734545 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b1dde-e7a2-407a-94a7-7ea3a6d54de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:20.735134 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b1dde-e7a2-407a-94a7-7ea3a6d54de2 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:20.735419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b1dde-e7a2-407a-94a7-7ea3a6d54de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:20.735693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b1dde-e7a2-407a-94a7-7ea3a6d54de2 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:20.736424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b1dde-e7a2-407a-94a7-7ea3a6d54de2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:20.736737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b1dde-e7a2-407a-94a7-7ea3a6d54de2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:20.738123 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b1dde-e7a2-407a-94a7-7ea3a6d54de2 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:20.751846 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 612/1238] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:20.758427 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe6e8e-3563-4cf1-ac61-6a17fa601404 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:20.897456 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe6e8e-3563-4cf1-ac61-6a17fa601404 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:20.937431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d3f46-3d9a-452a-b739-bc18e104f6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:20.938428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d3f46-3d9a-452a-b739-bc18e104f6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:20.938716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d3f46-3d9a-452a-b739-bc18e104f6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:20.938994 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d3f46-3d9a-452a-b739-bc18e104f6a7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:20.942076 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:20.948071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d3f46-3d9a-452a-b739-bc18e104f6a7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:20.950283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d3f46-3d9a-452a-b739-bc18e104f6a7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:20.966792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-d00d3f46-3d9a-452a-b739-bc18e104f6a7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 17:54:20.966792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:20.966792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:20.966792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:20.966792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:20.966792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:20.966792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:20.966792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:20.966792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:20.966792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:20.966792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:20.966792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:20.966792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:54:20.966792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:54:20.966792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:54:20.966792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:54:20.968005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:20.968005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:20.968005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:54:20.968005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:54:20.968005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:54:20.968005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:54:20.968005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:20.968005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:20.968005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:20.968005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:20.968005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:20.968005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:54:20.968005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:20.968005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:20.968005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:20.968005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:20.968005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:20.968005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:20.969337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:20.969337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:20.969337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:54:20.969337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:54:20.969337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:54:20.969337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:54:20.969337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:54:20.969337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:54:20.969337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 17:54:20.969337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:54:20.973195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 627/1239] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:54:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:20.974536 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b1dde-e7a2-407a-94a7-7ea3a6d54de2 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:21.090890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b1dde-e7a2-407a-94a7-7ea3a6d54de2 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:21.127896 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80369121-391e-45d8-9faa-99f77b378815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:21.131181 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80369121-391e-45d8-9faa-99f77b378815 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:21.131181 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80369121-391e-45d8-9faa-99f77b378815 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:21.131181 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80369121-391e-45d8-9faa-99f77b378815 None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:21.131181 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80369121-391e-45d8-9faa-99f77b378815 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:21.131181 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80369121-391e-45d8-9faa-99f77b378815 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:21.131812 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80369121-391e-45d8-9faa-99f77b378815 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:21.151286 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 613/1240] 10.222.0.76 () {56 vars in 1143 bytes} [Thu May 26 17:54:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:21.157381 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d3f46-3d9a-452a-b739-bc18e104f6a7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:21.285079 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d3f46-3d9a-452a-b739-bc18e104f6a7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:21.325204 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365f858-9578-46e2-bcff-c5458fb95f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:21.325862 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365f858-9578-46e2-bcff-c5458fb95f93 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:21.326151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365f858-9578-46e2-bcff-c5458fb95f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:21.326526 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365f858-9578-46e2-bcff-c5458fb95f93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:21.404804 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:21.413242 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365f858-9578-46e2-bcff-c5458fb95f93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:21.414766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365f858-9578-46e2-bcff-c5458fb95f93 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:21.540016 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 628/1241] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:54:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:54:21.541712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80369121-391e-45d8-9faa-99f77b378815 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:21.573635 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:21.663696 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80369121-391e-45d8-9faa-99f77b378815 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:21.713478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59181208-3180-4a4e-b694-f60472c6e23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:21.714524 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59181208-3180-4a4e-b694-f60472c6e23e None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:21.714950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59181208-3180-4a4e-b694-f60472c6e23e None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:21.715380 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59181208-3180-4a4e-b694-f60472c6e23e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:21.731551 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59181208-3180-4a4e-b694-f60472c6e23e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:21.734841 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59181208-3180-4a4e-b694-f60472c6e23e None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:21.754421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-59181208-3180-4a4e-b694-f60472c6e23e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:21.754421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:21.754421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:21.754421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:21.754421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:21.754421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:21.754421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:21.754421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:21.754421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:21.754421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:21.754421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:21.754421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:21.754421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:54:21.754421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:54:21.754421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:54:21.755491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:54:21.755491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:21.755491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:21.755491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:21.755491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:21.755491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:21.755491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:21.755491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:21.755491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:54:21.755491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:21.755491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:21.755491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:21.755491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:21.755491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:21.755491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:21.755491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:21.755491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:21.761412 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:54:21.761412 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:54:21.761412 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:21.761412 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:21.761412 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 614/1242] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:54:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:21.761799 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365f858-9578-46e2-bcff-c5458fb95f93 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:21.882586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:21.890927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365f858-9578-46e2-bcff-c5458fb95f93 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:21.938233 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5fd09-fe86-4a77-a960-26b0e98c9c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:21.939069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5fd09-fe86-4a77-a960-26b0e98c9c22 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:21.939229 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5fd09-fe86-4a77-a960-26b0e98c9c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:21.939543 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5fd09-fe86-4a77-a960-26b0e98c9c22 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:21.940402 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5fd09-fe86-4a77-a960-26b0e98c9c22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:21.940762 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5fd09-fe86-4a77-a960-26b0e98c9c22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:21.942506 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5fd09-fe86-4a77-a960-26b0e98c9c22 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:21.963113 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 629/1243] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:21.964547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59181208-3180-4a4e-b694-f60472c6e23e None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:22.056475 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:22.080623 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59181208-3180-4a4e-b694-f60472c6e23e None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:22.118752 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed963fa6-595b-4448-8116-09e8a1fabfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:22.119543 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed963fa6-595b-4448-8116-09e8a1fabfce None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:22.119831 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed963fa6-595b-4448-8116-09e8a1fabfce None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:22.120107 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed963fa6-595b-4448-8116-09e8a1fabfce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:22.127779 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed963fa6-595b-4448-8116-09e8a1fabfce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:22.129266 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed963fa6-595b-4448-8116-09e8a1fabfce None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:22.146309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-ed963fa6-595b-4448-8116-09e8a1fabfce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:54:22.146309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:22.146309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:22.146309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:22.146309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:22.146309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:22.146309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:22.146309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:22.146309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:22.146309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:22.146309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:22.146309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:22.146309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:54:22.146309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:54:22.146309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:54:22.147494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:54:22.147494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:22.147494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:22.147494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:22.147494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:22.147494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:22.147494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:22.147494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:22.147494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:54:22.147494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:22.147494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:22.147494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:22.147494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:22.147494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:22.147494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:22.147494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:22.147494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:22.148744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:54:22.148744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:54:22.148744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:54:22.148744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:54:22.148744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:54:22.148744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:54:22.148744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:54:22.148744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:22.148744 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 615/1244] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:54:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:54:22.153719 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5fd09-fe86-4a77-a960-26b0e98c9c22 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:22.268639 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5fd09-fe86-4a77-a960-26b0e98c9c22 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:22.308002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b6a10-75fd-4cc6-a261-1736ddcec854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:22.308586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b6a10-75fd-4cc6-a261-1736ddcec854 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:22.308897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b6a10-75fd-4cc6-a261-1736ddcec854 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:22.309202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b6a10-75fd-4cc6-a261-1736ddcec854 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:22.310097 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b6a10-75fd-4cc6-a261-1736ddcec854 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:22.310488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b6a10-75fd-4cc6-a261-1736ddcec854 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:22.312611 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b6a10-75fd-4cc6-a261-1736ddcec854 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:22.327419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 630/1245] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:54:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:22.334114 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed963fa6-595b-4448-8116-09e8a1fabfce None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:22.441171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed963fa6-595b-4448-8116-09e8a1fabfce None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:22.485094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b828c-5626-4d54-84e2-001a2d0b2857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:22.485772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b828c-5626-4d54-84e2-001a2d0b2857 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:22.486175 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b828c-5626-4d54-84e2-001a2d0b2857 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:22.486592 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b828c-5626-4d54-84e2-001a2d0b2857 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:22.487512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b828c-5626-4d54-84e2-001a2d0b2857 None admin] RBAC: Adding query filter params (role.id=e776303775a2423aa0c62c161ecb5a7e, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=596f126382e3440fb3cf7c357eef889f) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:22.487826 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b828c-5626-4d54-84e2-001a2d0b2857 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:22.489276 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b828c-5626-4d54-84e2-001a2d0b2857 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:22.496902 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 616/1246] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:54:22 2022] GET /identity/v3/role_assignments?user.id=596f126382e3440fb3cf7c357eef889f&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=e776303775a2423aa0c62c161ecb5a7e => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:25.059272 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86f921-1624-470e-b82f-d6daba63e862 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:25.059918 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86f921-1624-470e-b82f-d6daba63e862 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:25.059918 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86f921-1624-470e-b82f-d6daba63e862 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:25.061623 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 631/1247] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:54:25.067704 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288c8a4-cf76-47a7-bd68-ff647fe131f8 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:25.068164 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288c8a4-cf76-47a7-bd68-ff647fe131f8 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:25.068267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288c8a4-cf76-47a7-bd68-ff647fe131f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:25.074086 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:25.088419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-c288c8a4-cf76-47a7-bd68-ff647fe131f8 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:54:25.089627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-c288c8a4-cf76-47a7-bd68-ff647fe131f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:54:25.173135 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 617/1248] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:54:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:54:25.201102 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86f921-1624-470e-b82f-d6daba63e862 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:25.221561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:25.326367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86f921-1624-470e-b82f-d6daba63e862 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:25.365548 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9692bb7-e582-467c-8fc2-8bfb0374c859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:25.366433 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9692bb7-e582-467c-8fc2-8bfb0374c859 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:25.366732 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9692bb7-e582-467c-8fc2-8bfb0374c859 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:25.367045 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9692bb7-e582-467c-8fc2-8bfb0374c859 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:25.381081 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9692bb7-e582-467c-8fc2-8bfb0374c859 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:54:25.388714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c9692bb7-e582-467c-8fc2-8bfb0374c859 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:54:25.388714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:25.388714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:25.388714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:25.388714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:25.388714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:25.388714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:25.388714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:25.388714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:25.388714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:25.388714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:25.388714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:25.388714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:54:25.388714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:54:25.388714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:54:25.388714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:54:25.389929 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:54:25.389929 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:54:25.389929 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:54:25.389929 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:54:25.389929 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:25.389929 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:25.389929 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:25.389929 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:25.389929 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:25.389929 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:25.389929 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:25.389929 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:54:25.389929 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:25.389929 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:25.389929 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:25.389929 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:25.390967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:25.390967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:25.390967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:25.390967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:25.390967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:54:25.390967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:54:25.390967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:54:25.390967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:54:25.390967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:54:25.390967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:54:25.390967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:54:25.390967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:54:25.390967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 632/1249] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:54:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 17:54:25.396967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288c8a4-cf76-47a7-bd68-ff647fe131f8 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:25.516722 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288c8a4-cf76-47a7-bd68-ff647fe131f8 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:25.553987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:25.560055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c819a2-c626-42c8-a815-c3ce75ee7064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:25.561041 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c819a2-c626-42c8-a815-c3ce75ee7064 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:25.561420 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c819a2-c626-42c8-a815-c3ce75ee7064 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:25.562051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c819a2-c626-42c8-a815-c3ce75ee7064 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:25.562988 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c819a2-c626-42c8-a815-c3ce75ee7064 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:25.563340 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c819a2-c626-42c8-a815-c3ce75ee7064 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:25.565056 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c819a2-c626-42c8-a815-c3ce75ee7064 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:25.574873 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 618/1250] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:54:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:25.582455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9692bb7-e582-467c-8fc2-8bfb0374c859 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:25.676389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:25.709645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9692bb7-e582-467c-8fc2-8bfb0374c859 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:25.760616 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7db0f-6105-4f95-a63c-35c88a854273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:25.761475 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7db0f-6105-4f95-a63c-35c88a854273 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:25.761768 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7db0f-6105-4f95-a63c-35c88a854273 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:25.762050 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7db0f-6105-4f95-a63c-35c88a854273 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:25.773983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7db0f-6105-4f95-a63c-35c88a854273 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:25.775590 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7db0f-6105-4f95-a63c-35c88a854273 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:25.801958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-13a7db0f-6105-4f95-a63c-35c88a854273 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:25.801958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:25.801958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:25.801958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:25.801958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:25.801958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:25.801958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:25.801958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:25.801958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:25.801958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:25.801958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:25.801958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:25.801958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:54:25.801958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:54:25.801958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:54:25.803939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:54:25.803939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:25.803939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:25.803939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:25.803939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:25.803939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:25.803939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:25.803939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:25.803939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:54:25.803939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:25.803939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:25.803939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:25.803939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:25.803939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:25.803939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:25.803939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:25.803939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:25.806160 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:54:25.806160 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:54:25.806160 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:25.806160 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:54:25.806160 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 633/1251] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:54:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:25.813505 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c819a2-c626-42c8-a815-c3ce75ee7064 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:25.974051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c819a2-c626-42c8-a815-c3ce75ee7064 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:26.020036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6b288-6047-4c6b-8d63-9d2f7c08640f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:26.020709 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6b288-6047-4c6b-8d63-9d2f7c08640f None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:26.020997 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6b288-6047-4c6b-8d63-9d2f7c08640f None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:26.021278 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6b288-6047-4c6b-8d63-9d2f7c08640f None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:26.022012 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6b288-6047-4c6b-8d63-9d2f7c08640f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:26.022372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6b288-6047-4c6b-8d63-9d2f7c08640f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:26.023812 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6b288-6047-4c6b-8d63-9d2f7c08640f None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:26.038077 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 619/1252] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:54:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:54:26.044451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7db0f-6105-4f95-a63c-35c88a854273 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:26.173004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7db0f-6105-4f95-a63c-35c88a854273 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:26.205778 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:26.214889 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54533d98-1347-423c-bbd9-466be66b38e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:26.215715 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54533d98-1347-423c-bbd9-466be66b38e3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:26.216003 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54533d98-1347-423c-bbd9-466be66b38e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:26.216281 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54533d98-1347-423c-bbd9-466be66b38e3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:26.225191 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54533d98-1347-423c-bbd9-466be66b38e3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:26.226600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54533d98-1347-423c-bbd9-466be66b38e3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:26.242165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-54533d98-1347-423c-bbd9-466be66b38e3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 17:54:26.242165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:26.242165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:26.242165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:26.242165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:26.242165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:26.242165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:26.242165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:26.242165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:26.242165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:26.242165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:26.242165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:26.242165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:54:26.242165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:54:26.242165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:54:26.242165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:54:26.243726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:26.243726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:26.243726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:54:26.243726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:54:26.243726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:54:26.243726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:54:26.243726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:26.243726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:26.243726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:26.243726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:26.243726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:26.243726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:54:26.243726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:26.243726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:26.243726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:26.243726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:26.243726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:26.243726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:26.244939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:26.244939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:26.244939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:54:26.244939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:54:26.244939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:54:26.244939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:54:26.244939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:54:26.244939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:54:26.244939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 17:54:26.244939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:54:26.244939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 634/1253] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:54:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:26.249948 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6b288-6047-4c6b-8d63-9d2f7c08640f None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:26.313872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:26.368890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6b288-6047-4c6b-8d63-9d2f7c08640f None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:26.414000 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4a518-038e-4998-a19d-094157f3d8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:26.414733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4a518-038e-4998-a19d-094157f3d8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:26.415069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4a518-038e-4998-a19d-094157f3d8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:26.415370 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4a518-038e-4998-a19d-094157f3d8e7 None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:26.416311 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4a518-038e-4998-a19d-094157f3d8e7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:26.416664 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4a518-038e-4998-a19d-094157f3d8e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:26.418313 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4a518-038e-4998-a19d-094157f3d8e7 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:26.440743 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 620/1254] 10.222.0.76 () {56 vars in 1143 bytes} [Thu May 26 17:54:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:54:26.446889 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54533d98-1347-423c-bbd9-466be66b38e3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:26.573594 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54533d98-1347-423c-bbd9-466be66b38e3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:26.617952 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c63ca22-20f3-477e-a92f-28873d36d65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:26.618664 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63ca22-20f3-477e-a92f-28873d36d65c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:26.618990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63ca22-20f3-477e-a92f-28873d36d65c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:26.619341 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63ca22-20f3-477e-a92f-28873d36d65c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:26.691501 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:26.706536 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c63ca22-20f3-477e-a92f-28873d36d65c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:26.708232 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c63ca22-20f3-477e-a92f-28873d36d65c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:26.831723 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 635/1255] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:54:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 17:54:26.838120 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4a518-038e-4998-a19d-094157f3d8e7 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:26.978495 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4a518-038e-4998-a19d-094157f3d8e7 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:26.993716 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:27.022410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5860283-e4fe-4986-be74-7f996764ff28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:27.023271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5860283-e4fe-4986-be74-7f996764ff28 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:27.023584 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5860283-e4fe-4986-be74-7f996764ff28 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:27.023901 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5860283-e4fe-4986-be74-7f996764ff28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:27.036323 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5860283-e4fe-4986-be74-7f996764ff28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:27.037826 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5860283-e4fe-4986-be74-7f996764ff28 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:27.057466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-f5860283-e4fe-4986-be74-7f996764ff28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:27.057466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:27.057466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:27.057466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:27.057466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:27.057466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:27.057466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:27.057466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:27.057466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:27.057466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:27.057466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:27.057466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:27.057466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:54:27.057466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:54:27.057466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:54:27.058603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:54:27.058603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:27.058603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:27.058603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:27.058603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:27.058603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:27.058603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:27.058603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:27.058603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:54:27.058603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:27.058603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:27.058603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:27.058603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:27.058603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:27.058603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:27.058603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:27.058603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:27.059562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:54:27.059562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:54:27.059562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:27.059562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:27.059562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 621/1256] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:54:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:27.066093 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c63ca22-20f3-477e-a92f-28873d36d65c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:27.167323 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:27.180836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c63ca22-20f3-477e-a92f-28873d36d65c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:27.218456 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7fc52-20d9-444e-b635-4c5f0c8aa2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:27.219134 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7fc52-20d9-444e-b635-4c5f0c8aa2bb None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:27.219421 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7fc52-20d9-444e-b635-4c5f0c8aa2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:27.219699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7fc52-20d9-444e-b635-4c5f0c8aa2bb None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:27.220427 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a7fc52-20d9-444e-b635-4c5f0c8aa2bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:27.220733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a7fc52-20d9-444e-b635-4c5f0c8aa2bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:27.222120 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a7fc52-20d9-444e-b635-4c5f0c8aa2bb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:27.235608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 636/1257] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:27.241416 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5860283-e4fe-4986-be74-7f996764ff28 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:27.352310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5860283-e4fe-4986-be74-7f996764ff28 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:27.394656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99ccd1-cd9b-47f9-bae0-1019a4abd8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:27.395934 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99ccd1-cd9b-47f9-bae0-1019a4abd8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:27.396431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99ccd1-cd9b-47f9-bae0-1019a4abd8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:27.396911 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99ccd1-cd9b-47f9-bae0-1019a4abd8a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:27.408306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d99ccd1-cd9b-47f9-bae0-1019a4abd8a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:27.410016 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d99ccd1-cd9b-47f9-bae0-1019a4abd8a9 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:27.427345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-8d99ccd1-cd9b-47f9-bae0-1019a4abd8a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:54:27.427345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:27.427345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:27.427345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:27.427345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:27.427345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:27.427345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:27.427345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:27.427345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:27.427345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:27.427345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:27.427345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:27.427345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:54:27.427345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:54:27.427345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:54:27.434350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:54:27.434350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:27.434350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:27.434350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:27.434350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:27.434350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:27.434350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:27.434350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:27.434350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:54:27.434350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:27.434350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:27.434350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:27.434350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:27.434350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:27.434350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:27.434350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:27.434350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:27.435504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:54:27.435504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:54:27.435504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:54:27.435504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:54:27.435504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:54:27.435504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:54:27.435504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:54:27.435504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:27.435504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 622/1258] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:54:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:27.436588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7fc52-20d9-444e-b635-4c5f0c8aa2bb None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:27.554058 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7fc52-20d9-444e-b635-4c5f0c8aa2bb None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:27.593013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32ec8c-5721-41d2-a260-9eb91a8c91d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:27.593727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32ec8c-5721-41d2-a260-9eb91a8c91d2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:27.593974 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32ec8c-5721-41d2-a260-9eb91a8c91d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:27.594283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32ec8c-5721-41d2-a260-9eb91a8c91d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:27.595078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32ec8c-5721-41d2-a260-9eb91a8c91d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:27.595393 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32ec8c-5721-41d2-a260-9eb91a8c91d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:27.596907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32ec8c-5721-41d2-a260-9eb91a8c91d2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:27.611178 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 637/1259] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:54:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:54:27.617658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99ccd1-cd9b-47f9-bae0-1019a4abd8a9 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:27.645964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:27.736963 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99ccd1-cd9b-47f9-bae0-1019a4abd8a9 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:27.774182 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec7a2b-81b6-4db6-9c1f-bb1dc960ce54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:27.774980 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec7a2b-81b6-4db6-9c1f-bb1dc960ce54 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:27.775265 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec7a2b-81b6-4db6-9c1f-bb1dc960ce54 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:27.775542 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec7a2b-81b6-4db6-9c1f-bb1dc960ce54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:27.776556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec7a2b-81b6-4db6-9c1f-bb1dc960ce54 None admin] RBAC: Adding query filter params (role.id=4fa8f06085c741a5b3cf64e63cc0754f, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=596f126382e3440fb3cf7c357eef889f) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:27.776719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec7a2b-81b6-4db6-9c1f-bb1dc960ce54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:27.778220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec7a2b-81b6-4db6-9c1f-bb1dc960ce54 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:27.785677 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 623/1260] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:54:27 2022] GET /identity/v3/role_assignments?user.id=596f126382e3440fb3cf7c357eef889f&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=4fa8f06085c741a5b3cf64e63cc0754f => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:30.052029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44273b64-11bd-4dd4-b6dc-ff533ffa179c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:30.052594 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44273b64-11bd-4dd4-b6dc-ff533ffa179c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:30.052594 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44273b64-11bd-4dd4-b6dc-ff533ffa179c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:30.053971 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 638/1261] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:54:30.059563 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d363dd-349d-4be4-bf6a-d731a6808985 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:30.060010 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d363dd-349d-4be4-bf6a-d731a6808985 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:30.060138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d363dd-349d-4be4-bf6a-d731a6808985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:30.078737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-54d363dd-349d-4be4-bf6a-d731a6808985 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:54:30.079942 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-54d363dd-349d-4be4-bf6a-d731a6808985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:54:30.149823 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 624/1262] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:54:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 17:54:30.173239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44273b64-11bd-4dd4-b6dc-ff533ffa179c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:30.184567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:30.280365 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44273b64-11bd-4dd4-b6dc-ff533ffa179c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:30.316361 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b8e4fa-91ec-4e51-bedd-505065bbe19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:30.317066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8e4fa-91ec-4e51-bedd-505065bbe19c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:30.317347 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8e4fa-91ec-4e51-bedd-505065bbe19c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:30.317621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8e4fa-91ec-4e51-bedd-505065bbe19c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:30.328132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b8e4fa-91ec-4e51-bedd-505065bbe19c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:54:30.335555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-27b8e4fa-91ec-4e51-bedd-505065bbe19c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:54:30.335555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:30.335555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:30.335555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:30.335555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:30.335555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:30.335555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:30.335555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:30.335555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:30.335555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:30.335555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:30.335555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:30.335555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:54:30.335555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:54:30.335555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:54:30.335555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:54:30.336647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:54:30.336647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:54:30.336647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:54:30.336647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:54:30.336647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:30.336647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:30.336647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:30.336647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:30.336647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:30.336647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:30.336647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:30.336647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:54:30.336647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:30.336647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:30.336647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:30.336647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:30.337734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:30.337734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:30.337734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:30.337734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:30.337734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:54:30.337734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:54:30.337734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:54:30.337734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:54:30.337734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:54:30.337734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:54:30.337734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:54:30.337734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:54:30.337734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 639/1263] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:54:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:30.342827 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d363dd-349d-4be4-bf6a-d731a6808985 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:30.361188 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:30.442366 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d363dd-349d-4be4-bf6a-d731a6808985 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:30.477190 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca45c3f-a131-4ee5-b3df-ab397d743948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:30.477859 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca45c3f-a131-4ee5-b3df-ab397d743948 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:30.478145 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca45c3f-a131-4ee5-b3df-ab397d743948 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:30.478454 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca45c3f-a131-4ee5-b3df-ab397d743948 None admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:30.479200 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca45c3f-a131-4ee5-b3df-ab397d743948 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:30.479506 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca45c3f-a131-4ee5-b3df-ab397d743948 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:30.480867 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca45c3f-a131-4ee5-b3df-ab397d743948 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:30.489725 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 625/1264] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:54:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:30.495601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b8e4fa-91ec-4e51-bedd-505065bbe19c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:30.564852 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:30.599713 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b8e4fa-91ec-4e51-bedd-505065bbe19c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:30.633760 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb263ec1-ecc7-4fd3-8b91-0329be66a92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:30.634588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb263ec1-ecc7-4fd3-8b91-0329be66a92b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:30.634878 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb263ec1-ecc7-4fd3-8b91-0329be66a92b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:30.635161 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb263ec1-ecc7-4fd3-8b91-0329be66a92b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:30.645511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb263ec1-ecc7-4fd3-8b91-0329be66a92b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:30.646993 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb263ec1-ecc7-4fd3-8b91-0329be66a92b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:30.664525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-eb263ec1-ecc7-4fd3-8b91-0329be66a92b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:30.664525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:30.664525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:30.664525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:30.664525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:30.664525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:30.664525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:30.664525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:30.664525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:30.664525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:30.664525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:30.664525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:30.664525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:54:30.664525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:54:30.664525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:54:30.665544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:54:30.665544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:30.665544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:30.665544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:30.665544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:30.665544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:30.665544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:30.665544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:30.665544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:54:30.665544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:30.665544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:30.665544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:30.665544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:30.665544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:30.665544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:30.665544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:30.665544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:30.666692 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:54:30.666692 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:54:30.666692 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:30.666692 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:54:30.666692 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 640/1265] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:54:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:30.672179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca45c3f-a131-4ee5-b3df-ab397d743948 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:30.752166 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:30.772025 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca45c3f-a131-4ee5-b3df-ab397d743948 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:30.806541 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f887bcc-79d2-4ea0-a315-7f7d8975a65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:30.807206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f887bcc-79d2-4ea0-a315-7f7d8975a65b None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:30.807489 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f887bcc-79d2-4ea0-a315-7f7d8975a65b None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:30.807767 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f887bcc-79d2-4ea0-a315-7f7d8975a65b None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:30.808488 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f887bcc-79d2-4ea0-a315-7f7d8975a65b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:30.808799 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f887bcc-79d2-4ea0-a315-7f7d8975a65b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:30.810175 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f887bcc-79d2-4ea0-a315-7f7d8975a65b None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:30.828795 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 626/1266] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:54:30.830433 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb263ec1-ecc7-4fd3-8b91-0329be66a92b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:30.950439 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb263ec1-ecc7-4fd3-8b91-0329be66a92b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:30.971884 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:30.986423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dfe946-170d-4c70-a9d6-fb4428f40e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:30.987222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dfe946-170d-4c70-a9d6-fb4428f40e31 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:30.987500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dfe946-170d-4c70-a9d6-fb4428f40e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:30.987770 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dfe946-170d-4c70-a9d6-fb4428f40e31 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:30.995419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dfe946-170d-4c70-a9d6-fb4428f40e31 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:30.996862 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dfe946-170d-4c70-a9d6-fb4428f40e31 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:31.011559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-01dfe946-170d-4c70-a9d6-fb4428f40e31 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 17:54:31.011559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:31.011559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:31.011559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:31.011559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:31.011559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:31.011559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:31.011559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:31.011559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:31.011559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:31.011559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:31.011559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:31.011559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:54:31.011559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:54:31.011559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:54:31.011559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:54:31.012673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:31.012673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:31.012673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:54:31.012673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:54:31.012673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:54:31.012673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:54:31.012673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:31.012673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:31.012673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:31.012673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:31.012673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:31.012673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:54:31.012673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:31.012673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:31.012673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:31.012673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:31.012673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:31.012673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:31.014509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:31.014509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:31.014509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:54:31.014509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:54:31.014509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:54:31.014509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:54:31.014509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:54:31.014509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:54:31.014509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 17:54:31.014509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:54:31.014509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 641/1267] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:54:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:31.019301 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f887bcc-79d2-4ea0-a315-7f7d8975a65b None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:31.117847 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f887bcc-79d2-4ea0-a315-7f7d8975a65b None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:31.152848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc18ed-198b-41ce-8e27-543883a24e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:31.153525 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc18ed-198b-41ce-8e27-543883a24e8d None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:31.153805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc18ed-198b-41ce-8e27-543883a24e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:31.154074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc18ed-198b-41ce-8e27-543883a24e8d None admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:31.155734 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc18ed-198b-41ce-8e27-543883a24e8d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:31.156043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc18ed-198b-41ce-8e27-543883a24e8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:31.157480 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc18ed-198b-41ce-8e27-543883a24e8d None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:31.166682 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:31.175309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 627/1268] 10.222.0.76 () {56 vars in 1143 bytes} [Thu May 26 17:54:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:31.180909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dfe946-170d-4c70-a9d6-fb4428f40e31 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:31.284282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dfe946-170d-4c70-a9d6-fb4428f40e31 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:31.317428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af5ab0-c582-47e8-8aeb-2189ad08c8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:31.318228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af5ab0-c582-47e8-8aeb-2189ad08c8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:31.318517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af5ab0-c582-47e8-8aeb-2189ad08c8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:31.318789 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af5ab0-c582-47e8-8aeb-2189ad08c8d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:31.329641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af5ab0-c582-47e8-8aeb-2189ad08c8d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:31.331109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af5ab0-c582-47e8-8aeb-2189ad08c8d6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:31.347627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-82af5ab0-c582-47e8-8aeb-2189ad08c8d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:31.347627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:31.347627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:31.347627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:31.347627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:31.347627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:31.347627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:31.347627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:31.347627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:31.347627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:31.347627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:31.347627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:31.347627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:54:31.347627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:54:31.347627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:54:31.348811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:54:31.348811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:31.348811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:31.348811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:31.348811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:31.348811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:31.348811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:31.348811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:31.348811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:54:31.348811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:31.348811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:31.348811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:31.348811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:31.348811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:31.348811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:31.348811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:31.348811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:31.350039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:54:31.350039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:54:31.350039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:31.350039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:54:31.350039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 642/1269] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:54:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:31.354564 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc18ed-198b-41ce-8e27-543883a24e8d None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:31.457435 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc18ed-198b-41ce-8e27-543883a24e8d None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:31.492012 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8f02d-fd74-4811-9dd9-1618b4f7a898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:31.492677 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8f02d-fd74-4811-9dd9-1618b4f7a898 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:31.492957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8f02d-fd74-4811-9dd9-1618b4f7a898 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:31.493233 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8f02d-fd74-4811-9dd9-1618b4f7a898 None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:31.493950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8f02d-fd74-4811-9dd9-1618b4f7a898 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:31.494284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8f02d-fd74-4811-9dd9-1618b4f7a898 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:31.495654 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8f02d-fd74-4811-9dd9-1618b4f7a898 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:31.508069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 628/1270] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:31.513697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af5ab0-c582-47e8-8aeb-2189ad08c8d6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:31.546376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:31.617017 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af5ab0-c582-47e8-8aeb-2189ad08c8d6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:31.651889 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844e16f-2489-46ca-967a-cda0f9c90f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:31.652675 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844e16f-2489-46ca-967a-cda0f9c90f5d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:31.652956 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844e16f-2489-46ca-967a-cda0f9c90f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:31.653229 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844e16f-2489-46ca-967a-cda0f9c90f5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:31.660606 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844e16f-2489-46ca-967a-cda0f9c90f5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:31.662043 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844e16f-2489-46ca-967a-cda0f9c90f5d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:31.677308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-e844e16f-2489-46ca-967a-cda0f9c90f5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:54:31.677308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:31.677308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:31.677308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:31.677308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:31.677308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:31.677308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:31.677308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:31.677308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:31.677308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:31.677308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:31.677308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:31.677308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:54:31.677308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:54:31.677308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:54:31.678367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:54:31.678367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:31.678367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:31.678367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:31.678367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:31.678367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:31.678367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:31.678367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:31.678367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:54:31.678367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:31.678367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:31.678367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:31.678367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:31.678367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:31.678367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:31.678367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:31.678367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:31.679327 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:54:31.679327 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:54:31.679327 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:54:31.679327 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:54:31.679327 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:54:31.679327 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:54:31.679327 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:54:31.679327 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:54:31.679327 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 643/1271] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:54:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:31.684369 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8f02d-fd74-4811-9dd9-1618b4f7a898 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:31.743903 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:31.789681 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8f02d-fd74-4811-9dd9-1618b4f7a898 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:31.830455 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394813d-4d0a-4c8d-bdbc-81de606809da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:31.831212 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394813d-4d0a-4c8d-bdbc-81de606809da None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:31.831500 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394813d-4d0a-4c8d-bdbc-81de606809da None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:31.831801 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394813d-4d0a-4c8d-bdbc-81de606809da None admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:31.832874 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394813d-4d0a-4c8d-bdbc-81de606809da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:31.833182 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394813d-4d0a-4c8d-bdbc-81de606809da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:31.835124 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394813d-4d0a-4c8d-bdbc-81de606809da None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:31.850552 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 629/1272] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:54:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:31.856615 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844e16f-2489-46ca-967a-cda0f9c90f5d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:31.948967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:31.959027 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844e16f-2489-46ca-967a-cda0f9c90f5d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:31.995042 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372dd40-d67d-4235-ac57-820d95048895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:31.995963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372dd40-d67d-4235-ac57-820d95048895 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:31.996247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372dd40-d67d-4235-ac57-820d95048895 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:31.996525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372dd40-d67d-4235-ac57-820d95048895 None admin] PATH_INFO: `/v3/projects/ca1d09f491d944f99425a641561b03c3/users/596f126382e3440fb3cf7c357eef889f/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:31.999516 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7372dd40-d67d-4235-ac57-820d95048895 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1d09f491d944f99425a641561b03c3, user_id=596f126382e3440fb3cf7c357eef889f, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:32.000685 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7372dd40-d67d-4235-ac57-820d95048895 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:32.014773 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 644/1273] 10.222.0.76 () {58 vars in 1439 bytes} [Thu May 26 17:54:31 2022] PUT /identity/v3/projects/ca1d09f491d944f99425a641561b03c3/users/596f126382e3440fb3cf7c357eef889f/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 17:54:34.222396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880be34-5455-4427-9de9-17c1556ba3ee None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:34.222951 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880be34-5455-4427-9de9-17c1556ba3ee None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:34.222951 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880be34-5455-4427-9de9-17c1556ba3ee None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:34.224322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 630/1274] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:54:34.229585 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5037e-34dc-4c2f-baa4-ed3dc8128f2c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:34.230032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5037e-34dc-4c2f-baa4-ed3dc8128f2c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:34.230139 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5037e-34dc-4c2f-baa4-ed3dc8128f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:34.246912 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-24e5037e-34dc-4c2f-baa4-ed3dc8128f2c None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:54:34.248667 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-24e5037e-34dc-4c2f-baa4-ed3dc8128f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:54:34.319828 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 645/1275] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:54:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:54:34.344172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880be34-5455-4427-9de9-17c1556ba3ee None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:34.447511 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880be34-5455-4427-9de9-17c1556ba3ee None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:34.465309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:34.482609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8540e-d41e-4ddb-86c9-5f662a1de4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:34.483409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8540e-d41e-4ddb-86c9-5f662a1de4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:34.483696 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8540e-d41e-4ddb-86c9-5f662a1de4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:34.483972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8540e-d41e-4ddb-86c9-5f662a1de4c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:34.494424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8540e-d41e-4ddb-86c9-5f662a1de4c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 17:54:34.502950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-53a8540e-d41e-4ddb-86c9-5f662a1de4c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:54:34.502950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:34.502950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:34.502950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:34.502950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:34.502950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:34.502950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:34.502950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:34.502950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:34.502950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:34.502950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:34.502950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:34.502950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 17:54:34.502950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 17:54:34.502950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 17:54:34.502950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 17:54:34.504049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 17:54:34.504049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 17:54:34.504049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 17:54:34.504049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 17:54:34.504049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:34.504049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:34.504049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:34.504049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:34.504049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:34.504049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:34.504049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:34.504049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:54:34.504049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:34.504049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:34.504049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:34.504049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:34.505199 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:34.505199 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:34.505199 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:34.505199 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:34.505199 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 17:54:34.505199 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 17:54:34.505199 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 17:54:34.505199 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 17:54:34.505199 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 17:54:34.505199 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 17:54:34.505199 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 17:54:34.505199 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:34.505199 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 631/1276] 10.222.0.76 () {56 vars in 1096 bytes} [Thu May 26 17:54:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:34.510537 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5037e-34dc-4c2f-baa4-ed3dc8128f2c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:34.596413 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:34.616032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5037e-34dc-4c2f-baa4-ed3dc8128f2c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:34.649913 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e66dc-aa4f-4fd7-80e3-be4b0ead646e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:34.650608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e66dc-aa4f-4fd7-80e3-be4b0ead646e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:34.650898 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e66dc-aa4f-4fd7-80e3-be4b0ead646e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:34.651176 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e66dc-aa4f-4fd7-80e3-be4b0ead646e None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:34.651923 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e66dc-aa4f-4fd7-80e3-be4b0ead646e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:34.652239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e66dc-aa4f-4fd7-80e3-be4b0ead646e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:34.653634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e66dc-aa4f-4fd7-80e3-be4b0ead646e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:34.662915 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 646/1277] 10.222.0.76 () {56 vars in 1099 bytes} [Thu May 26 17:54:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:34.668787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8540e-d41e-4ddb-86c9-5f662a1de4c5 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:34.779674 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8540e-d41e-4ddb-86c9-5f662a1de4c5 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:34.818341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f1792-0093-4184-bd61-8a5d3d3d34a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:34.819278 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f1792-0093-4184-bd61-8a5d3d3d34a9 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:34.819646 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f1792-0093-4184-bd61-8a5d3d3d34a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:34.819908 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f1792-0093-4184-bd61-8a5d3d3d34a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:34.834956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f1792-0093-4184-bd61-8a5d3d3d34a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:34.836740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f1792-0093-4184-bd61-8a5d3d3d34a9 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:34.859876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-194f1792-0093-4184-bd61-8a5d3d3d34a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:34.859876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:34.859876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:34.859876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:34.859876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:34.859876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:34.859876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:34.859876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:34.859876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:34.859876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:34.859876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:34.859876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:34.859876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:54:34.859876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:54:34.859876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:54:34.860975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:54:34.860975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:34.860975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:34.860975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:34.860975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:34.860975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:34.860975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:34.860975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:34.860975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:54:34.860975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:34.860975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:34.860975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:34.860975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:34.860975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:34.860975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:34.860975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:34.860975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:34.861918 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:54:34.861918 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:54:34.861918 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:34.861918 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:34.861918 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 632/1278] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:54:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:34.868875 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e66dc-aa4f-4fd7-80e3-be4b0ead646e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:34.974385 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e66dc-aa4f-4fd7-80e3-be4b0ead646e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:35.008835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa080dbe-e474-4981-809e-321da6b6bcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:35.009502 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa080dbe-e474-4981-809e-321da6b6bcca None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:35.009791 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa080dbe-e474-4981-809e-321da6b6bcca None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:35.010071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa080dbe-e474-4981-809e-321da6b6bcca None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:35.010827 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa080dbe-e474-4981-809e-321da6b6bcca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:35.011137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa080dbe-e474-4981-809e-321da6b6bcca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:35.012521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa080dbe-e474-4981-809e-321da6b6bcca None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:35.024564 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 647/1279] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 17:54:35.031501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f1792-0093-4184-bd61-8a5d3d3d34a9 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:35.064365 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:35.135507 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f1792-0093-4184-bd61-8a5d3d3d34a9 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:35.169550 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645965a9-ecee-43aa-9532-c2afb1ba71a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:35.171052 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645965a9-ecee-43aa-9532-c2afb1ba71a6 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:35.171338 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645965a9-ecee-43aa-9532-c2afb1ba71a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:35.171618 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645965a9-ecee-43aa-9532-c2afb1ba71a6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:35.179386 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645965a9-ecee-43aa-9532-c2afb1ba71a6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:35.180830 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645965a9-ecee-43aa-9532-c2afb1ba71a6 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:35.195149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-645965a9-ecee-43aa-9532-c2afb1ba71a6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 17:54:35.195149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:35.195149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:35.195149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:35.195149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:35.195149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:35.195149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:35.195149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:35.195149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:35.195149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:35.195149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:35.195149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:35.195149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 17:54:35.195149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 17:54:35.195149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 17:54:35.195149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 17:54:35.196171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:35.196171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:35.196171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 17:54:35.196171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:54:35.196171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 17:54:35.196171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 17:54:35.196171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:35.196171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:35.196171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:35.196171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:35.196171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:35.196171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:54:35.196171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:35.196171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:35.196171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:35.196171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:35.196171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:35.196171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:35.197141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:35.197141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:35.197141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 17:54:35.197141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 17:54:35.197141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 17:54:35.197141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 17:54:35.197141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 17:54:35.197141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 17:54:35.197141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 17:54:35.197141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:35.197141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 633/1280] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:54:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:35.203179 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa080dbe-e474-4981-809e-321da6b6bcca None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:35.207265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:35.304332 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa080dbe-e474-4981-809e-321da6b6bcca None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:35.338982 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d90c4-ffee-4460-b7bf-f0a1e7703ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:35.339649 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d90c4-ffee-4460-b7bf-f0a1e7703ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:35.339936 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d90c4-ffee-4460-b7bf-f0a1e7703ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:35.340214 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d90c4-ffee-4460-b7bf-f0a1e7703ec5 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:35.341045 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425d90c4-ffee-4460-b7bf-f0a1e7703ec5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:35.341358 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425d90c4-ffee-4460-b7bf-f0a1e7703ec5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:35.343393 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425d90c4-ffee-4460-b7bf-f0a1e7703ec5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:35.361407 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 648/1281] 10.222.0.76 () {56 vars in 1143 bytes} [Thu May 26 17:54:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:35.367181 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645965a9-ecee-43aa-9532-c2afb1ba71a6 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:35.459413 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:35.470357 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645965a9-ecee-43aa-9532-c2afb1ba71a6 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:35.503097 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef237237-3cf6-443c-81f3-a795f246c11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:35.503746 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef237237-3cf6-443c-81f3-a795f246c11a None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:35.504034 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef237237-3cf6-443c-81f3-a795f246c11a None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:35.504316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef237237-3cf6-443c-81f3-a795f246c11a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:35.571452 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef237237-3cf6-443c-81f3-a795f246c11a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:35.572857 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef237237-3cf6-443c-81f3-a795f246c11a None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:35.661821 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:35.672471 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 634/1282] 10.222.0.76 () {58 vars in 1282 bytes} [Thu May 26 17:54:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 17:54:35.678607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d90c4-ffee-4460-b7bf-f0a1e7703ec5 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:35.740943 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:35.781864 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d90c4-ffee-4460-b7bf-f0a1e7703ec5 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:35.819798 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9092c0-0b48-41d3-8faf-6ce8eee698e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:35.820681 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9092c0-0b48-41d3-8faf-6ce8eee698e9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:35.820936 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9092c0-0b48-41d3-8faf-6ce8eee698e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:35.821303 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9092c0-0b48-41d3-8faf-6ce8eee698e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:35.832541 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9092c0-0b48-41d3-8faf-6ce8eee698e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:35.833917 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9092c0-0b48-41d3-8faf-6ce8eee698e9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:35.851353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-1f9092c0-0b48-41d3-8faf-6ce8eee698e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:35.851353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:35.851353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:35.851353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:35.851353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:35.851353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:35.851353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:35.851353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:35.851353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:35.851353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:35.851353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:35.851353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:35.851353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 17:54:35.851353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 17:54:35.851353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 17:54:35.852504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 17:54:35.852504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:35.852504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:35.852504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:35.852504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:35.852504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:35.852504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:35.852504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:35.852504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:54:35.852504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:35.852504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:35.852504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:35.852504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:35.852504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:35.852504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:35.852504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:35.852504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:35.853641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 17:54:35.853641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 17:54:35.853641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 17:54:35.853641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:54:35.853641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 649/1283] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:54:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:35.858949 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef237237-3cf6-443c-81f3-a795f246c11a None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:35.956831 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef237237-3cf6-443c-81f3-a795f246c11a None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:35.990176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d814e0-6b0f-4bd5-a722-632874b091af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:35.990856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d814e0-6b0f-4bd5-a722-632874b091af None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:35.991135 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d814e0-6b0f-4bd5-a722-632874b091af None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:35.991406 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d814e0-6b0f-4bd5-a722-632874b091af None admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:35.992118 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d814e0-6b0f-4bd5-a722-632874b091af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:35.992428 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d814e0-6b0f-4bd5-a722-632874b091af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:35.993777 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d814e0-6b0f-4bd5-a722-632874b091af None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:36.005802 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 635/1284] 10.222.0.76 () {56 vars in 1109 bytes} [Thu May 26 17:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:36.011621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9092c0-0b48-41d3-8faf-6ce8eee698e9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:36.109338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9092c0-0b48-41d3-8faf-6ce8eee698e9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:36.122864 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:36.143553 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3b063-12ab-4a8c-9c32-acc21d0cdae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:36.144349 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3b063-12ab-4a8c-9c32-acc21d0cdae3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:36.144631 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3b063-12ab-4a8c-9c32-acc21d0cdae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:36.144907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3b063-12ab-4a8c-9c32-acc21d0cdae3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:36.151644 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab3b063-12ab-4a8c-9c32-acc21d0cdae3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:36.153108 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab3b063-12ab-4a8c-9c32-acc21d0cdae3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:36.166843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-dab3b063-12ab-4a8c-9c32-acc21d0cdae3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:54:36.166843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:36.166843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:36.166843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:36.166843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:36.166843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:36.166843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:36.166843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:36.166843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:36.166843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:36.166843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:36.166843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:36.166843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 17:54:36.166843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 17:54:36.166843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 17:54:36.167895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 17:54:36.167895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:36.167895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:36.167895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:36.167895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:36.167895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:36.167895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:36.167895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:36.167895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( May 26 17:54:36.167895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:36.167895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:36.167895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:36.167895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:36.167895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:36.167895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:36.167895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:36.167895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:36.169030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 17:54:36.169030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 17:54:36.169030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 17:54:36.169030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 17:54:36.169030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 17:54:36.169030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 17:54:36.169030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 17:54:36.169030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application May 26 17:54:36.169030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 650/1285] 10.222.0.76 () {56 vars in 1111 bytes} [Thu May 26 17:54:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:36.174657 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d814e0-6b0f-4bd5-a722-632874b091af None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:36.225287 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:36.275577 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d814e0-6b0f-4bd5-a722-632874b091af None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:36.309047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543fcf1-7c4e-41c1-88ac-532aa3e90006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:36.309699 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543fcf1-7c4e-41c1-88ac-532aa3e90006 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:36.309979 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543fcf1-7c4e-41c1-88ac-532aa3e90006 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:36.310283 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543fcf1-7c4e-41c1-88ac-532aa3e90006 None admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:36.311633 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543fcf1-7c4e-41c1-88ac-532aa3e90006 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:36.311929 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543fcf1-7c4e-41c1-88ac-532aa3e90006 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:36.313355 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543fcf1-7c4e-41c1-88ac-532aa3e90006 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:36.325330 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 636/1286] 10.222.0.76 () {56 vars in 1148 bytes} [Thu May 26 17:54:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:36.331677 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3b063-12ab-4a8c-9c32-acc21d0cdae3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:36.433058 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3b063-12ab-4a8c-9c32-acc21d0cdae3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:36.468523 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269dfc4-fd23-4cd8-ae5c-9965a006b035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:36.469176 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269dfc4-fd23-4cd8-ae5c-9965a006b035 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:36.469463 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269dfc4-fd23-4cd8-ae5c-9965a006b035 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:36.469741 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269dfc4-fd23-4cd8-ae5c-9965a006b035 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:36.470605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269dfc4-fd23-4cd8-ae5c-9965a006b035 None admin] RBAC: Adding query filter params (role.id=4fa8f06085c741a5b3cf64e63cc0754f, scope.project.id=ca1d09f491d944f99425a641561b03c3, user.id=596f126382e3440fb3cf7c357eef889f) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:36.470922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269dfc4-fd23-4cd8-ae5c-9965a006b035 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:36.472371 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269dfc4-fd23-4cd8-ae5c-9965a006b035 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:36.478409 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 651/1287] 10.222.0.76 () {56 vars in 1374 bytes} [Thu May 26 17:54:36 2022] GET /identity/v3/role_assignments?user.id=596f126382e3440fb3cf7c357eef889f&scope.project.id=ca1d09f491d944f99425a641561b03c3&role.id=4fa8f06085c741a5b3cf64e63cc0754f => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:38.720772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a6126-0f92-4186-b92a-966f9b742efe None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:38.721337 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a6126-0f92-4186-b92a-966f9b742efe None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:38.721337 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a6126-0f92-4186-b92a-966f9b742efe None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:38.722733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 637/1288] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:54:38.727927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf0e31-7178-41cf-824a-cb1e490694a2 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:38.728382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf0e31-7178-41cf-824a-cb1e490694a2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:38.728488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf0e31-7178-41cf-824a-cb1e490694a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:38.745108 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d9cf0e31-7178-41cf-824a-cb1e490694a2 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:54:38.746412 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d9cf0e31-7178-41cf-824a-cb1e490694a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:54:38.777048 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:38.817917 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 652/1289] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:54:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:54:38.843498 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a6126-0f92-4186-b92a-966f9b742efe None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:38.954499 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a6126-0f92-4186-b92a-966f9b742efe None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:38.965388 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:38.989164 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75883522-9478-41f6-9131-49c21e89a5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:38.989963 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75883522-9478-41f6-9131-49c21e89a5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:38.990281 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75883522-9478-41f6-9131-49c21e89a5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:38.990563 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75883522-9478-41f6-9131-49c21e89a5e9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:38.991213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75883522-9478-41f6-9131-49c21e89a5e9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:38.992580 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75883522-9478-41f6-9131-49c21e89a5e9 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:39.004909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-75883522-9478-41f6-9131-49c21e89a5e9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 17:54:39.004909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:39.004909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:54:39.004909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:54:39.004909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:54:39.004909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:54:39.004909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:54:39.004909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:54:39.004909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 17:54:39.004909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:39.004909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:54:39.004909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:39.004909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:39.004909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:39.004909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:39.004909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:39.004909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:39.006244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:39.006244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:39.006244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:39.006244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:39.006244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:39.006244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:39.006244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:54:39.006244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:54:39.006244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:54:39.006244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:54:39.006244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:39.006244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:39.006244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:39.006244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:39.006244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:39.006244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:39.007485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:39.007485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:54:39.007485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:39.007485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:39.007485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:39.007485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:39.007485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:39.007485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:39.007485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:39.007485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:39.007485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:54:39.007485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:54:39.007485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 17:54:39.007485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:39.007485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 638/1290] 10.222.0.76 () {56 vars in 1117 bytes} [Thu May 26 17:54:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:39.012242 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf0e31-7178-41cf-824a-cb1e490694a2 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:39.111871 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf0e31-7178-41cf-824a-cb1e490694a2 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:39.149732 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e54fd-bc42-407f-b02b-b0024e6d499d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:39.150449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e54fd-bc42-407f-b02b-b0024e6d499d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:39.150735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e54fd-bc42-407f-b02b-b0024e6d499d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:39.151011 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e54fd-bc42-407f-b02b-b0024e6d499d None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:39.151726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e54fd-bc42-407f-b02b-b0024e6d499d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:39.152032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e54fd-bc42-407f-b02b-b0024e6d499d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:39.153422 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e54fd-bc42-407f-b02b-b0024e6d499d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:39.158942 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 653/1291] 10.222.0.76 () {56 vars in 1116 bytes} [Thu May 26 17:54:39 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:39.165151 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75883522-9478-41f6-9131-49c21e89a5e9 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:39.264534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75883522-9478-41f6-9131-49c21e89a5e9 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:39.297990 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247772e0-7e81-4edb-a0bc-4c344fd6ea2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:39.298682 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247772e0-7e81-4edb-a0bc-4c344fd6ea2e None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:39.298968 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247772e0-7e81-4edb-a0bc-4c344fd6ea2e None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:39.299247 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247772e0-7e81-4edb-a0bc-4c344fd6ea2e None admin] PATH_INFO: `/v3/services` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:39.299952 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247772e0-7e81-4edb-a0bc-4c344fd6ea2e None admin] RBAC: Adding query filter params (type=placement) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:39.300260 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247772e0-7e81-4edb-a0bc-4c344fd6ea2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:39.301646 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247772e0-7e81-4edb-a0bc-4c344fd6ea2e None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:39.306385 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 639/1292] 10.222.0.76 () {56 vars in 1116 bytes} [Thu May 26 17:54:39 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:41.510215 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5767943-75d5-4dd2-b870-2bc1a0cc9f7b None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:41.510753 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5767943-75d5-4dd2-b870-2bc1a0cc9f7b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:41.510847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5767943-75d5-4dd2-b870-2bc1a0cc9f7b None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:41.512176 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 654/1293] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:54:41.517396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f3eef-fb13-4a23-8dbf-3b2719123e1a None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:41.517836 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f3eef-fb13-4a23-8dbf-3b2719123e1a None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:41.517953 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f3eef-fb13-4a23-8dbf-3b2719123e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:41.535453 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-ac2f3eef-fb13-4a23-8dbf-3b2719123e1a None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:54:41.536763 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-ac2f3eef-fb13-4a23-8dbf-3b2719123e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:54:41.572138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:41.611203 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 640/1294] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:54:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:54:41.634911 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5767943-75d5-4dd2-b870-2bc1a0cc9f7b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:41.665845 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:41.738756 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5767943-75d5-4dd2-b870-2bc1a0cc9f7b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:41.773738 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1a0be-84c6-489a-8d03-fc08c9b0d770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:41.774352 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1a0be-84c6-489a-8d03-fc08c9b0d770 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:41.774642 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1a0be-84c6-489a-8d03-fc08c9b0d770 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:41.774923 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1a0be-84c6-489a-8d03-fc08c9b0d770 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:41.775657 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1a0be-84c6-489a-8d03-fc08c9b0d770 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:41.777032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1a0be-84c6-489a-8d03-fc08c9b0d770 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:41.791622 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 655/1295] 10.222.0.76 () {60 vars in 1141 bytes} [Thu May 26 17:54:41 2022] POST /identity/v3/services => generated 246 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:54:44.058635 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe2abd-5f22-4286-a72a-d772a6d69534 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:44.059166 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe2abd-5f22-4286-a72a-d772a6d69534 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:44.059290 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe2abd-5f22-4286-a72a-d772a6d69534 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:44.061067 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 641/1296] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:54:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:54:44.066558 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e09d2c-2b34-4290-95af-ce5dfd75300a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:44.067045 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e09d2c-2b34-4290-95af-ce5dfd75300a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:44.067143 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e09d2c-2b34-4290-95af-ce5dfd75300a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:44.085152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-18e09d2c-2b34-4290-95af-ce5dfd75300a None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:54:44.086387 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-18e09d2c-2b34-4290-95af-ce5dfd75300a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:54:44.166547 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 656/1297] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:54:44 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:54:44.190729 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe2abd-5f22-4286-a72a-d772a6d69534 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:44.299159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe2abd-5f22-4286-a72a-d772a6d69534 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:44.335446 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d8590-1386-4bdf-8800-c99839222da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:44.336171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d8590-1386-4bdf-8800-c99839222da3 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:44.336451 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d8590-1386-4bdf-8800-c99839222da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:44.336725 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d8590-1386-4bdf-8800-c99839222da3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:44.337388 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d8590-1386-4bdf-8800-c99839222da3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:44.338867 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d8590-1386-4bdf-8800-c99839222da3 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:44.352310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-693d8590-1386-4bdf-8800-c99839222da3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 17:54:44.352310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:44.352310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:54:44.352310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:54:44.352310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:54:44.352310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:54:44.352310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:54:44.352310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:54:44.352310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 17:54:44.352310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:44.352310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:54:44.352310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:44.352310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:44.352310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:44.352310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:44.352310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:44.352310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:44.353621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:44.353621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:44.353621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:44.353621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:44.353621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:44.353621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:44.353621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:54:44.353621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:54:44.353621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:54:44.353621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:54:44.353621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:44.353621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:44.353621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:44.353621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:44.353621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:44.353621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:44.354537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:44.354537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:54:44.354537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:44.354537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:44.354537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:44.354537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:44.354537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:44.354537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:44.354537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:44.354537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:44.354537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:54:44.354537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:54:44.354537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 17:54:44.354537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:44.354537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 642/1298] 10.222.0.76 () {56 vars in 1117 bytes} [Thu May 26 17:54:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:44.361386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e09d2c-2b34-4290-95af-ce5dfd75300a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:44.381625 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:44.468853 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e09d2c-2b34-4290-95af-ce5dfd75300a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:44.506530 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffb4b6-c89e-49a1-a488-b3c28ad436f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:44.507211 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffb4b6-c89e-49a1-a488-b3c28ad436f4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:44.507499 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffb4b6-c89e-49a1-a488-b3c28ad436f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:44.507779 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffb4b6-c89e-49a1-a488-b3c28ad436f4 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:44.508492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ffb4b6-c89e-49a1-a488-b3c28ad436f4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:44.508800 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ffb4b6-c89e-49a1-a488-b3c28ad436f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:44.510221 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ffb4b6-c89e-49a1-a488-b3c28ad436f4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:44.516135 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 657/1299] 10.222.0.76 () {56 vars in 1116 bytes} [Thu May 26 17:54:44 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:44.522444 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d8590-1386-4bdf-8800-c99839222da3 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:44.555269 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:44.624880 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d8590-1386-4bdf-8800-c99839222da3 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:44.657712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521d776-dd24-4c22-86f4-269d7cdd9290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:44.658400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521d776-dd24-4c22-86f4-269d7cdd9290 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:44.658681 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521d776-dd24-4c22-86f4-269d7cdd9290 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:44.658957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521d776-dd24-4c22-86f4-269d7cdd9290 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:44.659754 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521d776-dd24-4c22-86f4-269d7cdd9290 None admin] RBAC: Adding query filter params (interface=public, service_id=e4af1c70d78342a9b75f0ae27f09c4ce, region_id=RegionOne) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:44.660308 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521d776-dd24-4c22-86f4-269d7cdd9290 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:44.661861 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521d776-dd24-4c22-86f4-269d7cdd9290 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:44.667915 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 643/1300] 10.222.0.76 () {56 vars in 1251 bytes} [Thu May 26 17:54:44 2022] GET /identity/v3/endpoints?service_id=e4af1c70d78342a9b75f0ae27f09c4ce&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:44.674560 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffb4b6-c89e-49a1-a488-b3c28ad436f4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:44.758818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:44.777296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffb4b6-c89e-49a1-a488-b3c28ad436f4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:44.814611 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b1b64-6607-49a1-ba9d-aa6acf645a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:44.815278 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b1b64-6607-49a1-ba9d-aa6acf645a0b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:44.815565 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b1b64-6607-49a1-ba9d-aa6acf645a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:44.815843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b1b64-6607-49a1-ba9d-aa6acf645a0b None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:44.816523 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b1b64-6607-49a1-ba9d-aa6acf645a0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:44.817907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b1b64-6607-49a1-ba9d-aa6acf645a0b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:44.824789 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 658/1301] 10.222.0.76 () {56 vars in 1087 bytes} [Thu May 26 17:54:44 2022] GET /identity/v3/services => generated 2191 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:54:47.057974 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef3fe6-d2ce-4bb5-8d96-b180d4f80b3f None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:47.058538 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef3fe6-d2ce-4bb5-8d96-b180d4f80b3f None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:47.058659 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef3fe6-d2ce-4bb5-8d96-b180d4f80b3f None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:47.059929 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 644/1302] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:54:47.065259 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef95ac4-da2b-4a47-8324-4cbbe76c9ac7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:47.065669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef95ac4-da2b-4a47-8324-4cbbe76c9ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:47.065824 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef95ac4-da2b-4a47-8324-4cbbe76c9ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:47.082301 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4ef95ac4-da2b-4a47-8324-4cbbe76c9ac7 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:54:47.083480 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4ef95ac4-da2b-4a47-8324-4cbbe76c9ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:54:47.150850 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 659/1303] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:54:47 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 17:54:47.175508 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef3fe6-d2ce-4bb5-8d96-b180d4f80b3f None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:47.271089 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:47.279701 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef3fe6-d2ce-4bb5-8d96-b180d4f80b3f None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:47.313603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7ac9d-bea1-47bb-afd9-c2b582ac6176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:47.314361 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7ac9d-bea1-47bb-afd9-c2b582ac6176 None admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:47.314647 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7ac9d-bea1-47bb-afd9-c2b582ac6176 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:47.314923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7ac9d-bea1-47bb-afd9-c2b582ac6176 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:47.315572 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc7ac9d-bea1-47bb-afd9-c2b582ac6176 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:47.316937 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc7ac9d-bea1-47bb-afd9-c2b582ac6176 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:47.329478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application [None req-cdc7ac9d-bea1-47bb-afd9-c2b582ac6176 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 17:54:47.329478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:47.329478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 17:54:47.329478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 17:54:47.329478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 17:54:47.329478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 17:54:47.329478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 17:54:47.329478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:54:47.329478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 17:54:47.329478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:47.329478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 17:54:47.329478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:47.329478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 17:54:47.329478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 17:54:47.329478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 17:54:47.329478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 17:54:47.329478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 17:54:47.330760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 17:54:47.330760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 17:54:47.330760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 17:54:47.330760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 17:54:47.330760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 17:54:47.330760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 17:54:47.330760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 17:54:47.330760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 17:54:47.330760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 17:54:47.330760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 17:54:47.330760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 17:54:47.330760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 17:54:47.330760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 17:54:47.330760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 17:54:47.330760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 17:54:47.330760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.get_or_create( May 26 17:54:47.331620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 17:54:47.331620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application with Lock( May 26 17:54:47.331620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 17:54:47.331620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self._enter() May 26 17:54:47.331620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 17:54:47.331620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 17:54:47.331620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 17:54:47.331620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application return self.creator() May 26 17:54:47.331620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 17:54:47.331620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application created_value = creator( May 26 17:54:47.331620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 17:54:47.331620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 17:54:47.331620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 17:54:47.331620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: ERROR keystone.server.flask.application May 26 17:54:47.331620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 645/1304] 10.222.0.76 () {56 vars in 1117 bytes} [Thu May 26 17:54:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 17:54:47.336931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef95ac4-da2b-4a47-8324-4cbbe76c9ac7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:47.414305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:47.437890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef95ac4-da2b-4a47-8324-4cbbe76c9ac7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:47.472562 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd10c70-f047-4cb2-9eed-d681d5db52c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:47.473228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd10c70-f047-4cb2-9eed-d681d5db52c2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:47.473516 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd10c70-f047-4cb2-9eed-d681d5db52c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:47.473794 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd10c70-f047-4cb2-9eed-d681d5db52c2 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:47.474527 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd10c70-f047-4cb2-9eed-d681d5db52c2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 17:54:47.474836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd10c70-f047-4cb2-9eed-d681d5db52c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:47.476213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd10c70-f047-4cb2-9eed-d681d5db52c2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:47.481409 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 660/1305] 10.222.0.76 () {56 vars in 1116 bytes} [Thu May 26 17:54:47 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 17:54:47.487223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7ac9d-bea1-47bb-afd9-c2b582ac6176 None admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:54:47.588439 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7ac9d-bea1-47bb-afd9-c2b582ac6176 None admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:54:47.623051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc7c59-65cf-424e-b898-b30da6f4f0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:54:47.623717 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc7c59-65cf-424e-b898-b30da6f4f0e3 None admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:54:47.624007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc7c59-65cf-424e-b898-b30da6f4f0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:54:47.624287 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc7c59-65cf-424e-b898-b30da6f4f0e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:54:47.625028 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc7c59-65cf-424e-b898-b30da6f4f0e3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:54:47.626432 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc7c59-65cf-424e-b898-b30da6f4f0e3 None admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:54:47.645102 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 646/1306] 10.222.0.76 () {60 vars in 1144 bytes} [Thu May 26 17:54:47 2022] POST /identity/v3/endpoints => generated 331 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 17:54:57.478452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=30 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:54:57.643650 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:55:53.411666 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe0971-7089-4eb8-8c17-b42e25ebd00d None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:53.412214 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe0971-7089-4eb8-8c17-b42e25ebd00d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:53.412214 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe0971-7089-4eb8-8c17-b42e25ebd00d None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:53.419739 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6feb3-6897-4f68-9f74-6065e6be78ce None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:53.420216 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6feb3-6897-4f68-9f74-6065e6be78ce None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:53.420297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6feb3-6897-4f68-9f74-6065e6be78ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:53.423865 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 661/1307] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:55:53 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:55:53.510661 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-fae6feb3-6897-4f68-9f74-6065e6be78ce None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:55:53.512360 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-fae6feb3-6897-4f68-9f74-6065e6be78ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:55:53.782243 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 647/1308] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:55:53.861656 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618c028-beed-489e-ad82-e55c07d4908a None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:53.862294 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618c028-beed-489e-ad82-e55c07d4908a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:53.862873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618c028-beed-489e-ad82-e55c07d4908a None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:53.864472 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 662/1309] 10.222.0.76 () {54 vars in 964 bytes} [Thu May 26 17:55:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:55:53.874418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae549216-f0ca-4561-b62d-376a10a24bef None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:53.874778 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae549216-f0ca-4561-b62d-376a10a24bef None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:53.875143 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae549216-f0ca-4561-b62d-376a10a24bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:53.988443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-ae549216-f0ca-4561-b62d-376a10a24bef None None] MFA Rules not processed for user `8df66990a3734209a350c8b6768d259f`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:55:53.989974 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-ae549216-f0ca-4561-b62d-376a10a24bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:55:54.015669 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:55:54.096403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 648/1310] 10.222.0.76 () {58 vars in 1061 bytes} [Thu May 26 17:55:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:55:54.106282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2618c028-beed-489e-ad82-e55c07d4908a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:55:54.108481 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2618c028-beed-489e-ad82-e55c07d4908a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:55:54.248158 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2618c028-beed-489e-ad82-e55c07d4908a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:55:54.347285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39bd5ce-b336-48a3-a008-1ac8a16d05d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:55:54.347951 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39bd5ce-b336-48a3-a008-1ac8a16d05d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:54.348239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39bd5ce-b336-48a3-a008-1ac8a16d05d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:54.348518 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39bd5ce-b336-48a3-a008-1ac8a16d05d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:54.350216 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d39bd5ce-b336-48a3-a008-1ac8a16d05d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:55:54.412829 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:55:54.475769 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39bd5ce-b336-48a3-a008-1ac8a16d05d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:55:54.477141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39bd5ce-b336-48a3-a008-1ac8a16d05d2 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:55:54.498795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 663/1311] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:55:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:55:57.087795 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4173459-2cb4-4a97-985e-5538d3d543b6 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:57.088277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4173459-2cb4-4a97-985e-5538d3d543b6 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:57.088464 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4173459-2cb4-4a97-985e-5538d3d543b6 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:57.094721 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 649/1312] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:55:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:55:57.096218 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624eb951-c0f9-4ce6-be68-a292a0443c3f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:57.096648 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624eb951-c0f9-4ce6-be68-a292a0443c3f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:57.096779 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624eb951-c0f9-4ce6-be68-a292a0443c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:57.175248 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-624eb951-c0f9-4ce6-be68-a292a0443c3f None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:55:57.176476 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-624eb951-c0f9-4ce6-be68-a292a0443c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:55:57.278843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 664/1313] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:55:57.316346 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4173459-2cb4-4a97-985e-5538d3d543b6 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:55:57.317907 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-d4173459-2cb4-4a97-985e-5538d3d543b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:55:57.424311 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4173459-2cb4-4a97-985e-5538d3d543b6 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:55:57.497302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:55:57.500291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d6f23e-0f0a-4b1e-bfc1-891561451f09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:55:57.501125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6f23e-0f0a-4b1e-bfc1-891561451f09 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:55:57.501560 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6f23e-0f0a-4b1e-bfc1-891561451f09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:55:57.501956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6f23e-0f0a-4b1e-bfc1-891561451f09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:55:57.503915 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-96d6f23e-0f0a-4b1e-bfc1-891561451f09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:55:57.618781 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6f23e-0f0a-4b1e-bfc1-891561451f09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:55:57.620387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6f23e-0f0a-4b1e-bfc1-891561451f09 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:55:57.640048 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 650/1314] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:55:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:56:00.226709 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ae8cd-654d-4969-8d3e-1dc0fc06b904 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:00.227270 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ae8cd-654d-4969-8d3e-1dc0fc06b904 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:00.227270 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ae8cd-654d-4969-8d3e-1dc0fc06b904 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:00.228801 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 665/1315] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:56:00.234646 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc2a75-9b09-4ffc-92a2-d1d9300d3d19 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:00.235169 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc2a75-9b09-4ffc-92a2-d1d9300d3d19 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:00.235169 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc2a75-9b09-4ffc-92a2-d1d9300d3d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:00.306698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-e3dc2a75-9b09-4ffc-92a2-d1d9300d3d19 None None] MFA Rules not processed for user `95b4e2fbe1aa4fa5b7cebed5b01639cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:56:00.308030 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-e3dc2a75-9b09-4ffc-92a2-d1d9300d3d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:00.366766 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:56:00.414310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 651/1316] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:56:00 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:56:00.455252 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ae8cd-654d-4969-8d3e-1dc0fc06b904 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:56:00.456817 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ef4ae8cd-654d-4969-8d3e-1dc0fc06b904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:00.467379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:56:00.570624 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ae8cd-654d-4969-8d3e-1dc0fc06b904 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:56:00.646105 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88daa17-3feb-4d63-8238-01e8a4753f01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:56:00.646792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88daa17-3feb-4d63-8238-01e8a4753f01 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:00.647073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88daa17-3feb-4d63-8238-01e8a4753f01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:00.647348 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88daa17-3feb-4d63-8238-01e8a4753f01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:00.649009 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d88daa17-3feb-4d63-8238-01e8a4753f01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:00.714006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:56:00.763142 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88daa17-3feb-4d63-8238-01e8a4753f01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:56:00.764643 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88daa17-3feb-4d63-8238-01e8a4753f01 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:56:00.785236 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 666/1317] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:56:00 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:56:03.639723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb5472-7ad0-4354-8661-9f8d15d199fe None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:03.640294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb5472-7ad0-4354-8661-9f8d15d199fe None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:03.640390 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb5472-7ad0-4354-8661-9f8d15d199fe None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:03.641842 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 652/1318] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:56:03.648207 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1840bc-1155-414a-bd1f-2e30695fadeb None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:03.648745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1840bc-1155-414a-bd1f-2e30695fadeb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:03.648745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1840bc-1155-414a-bd1f-2e30695fadeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:03.721380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-bf1840bc-1155-414a-bd1f-2e30695fadeb None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:56:03.722637 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bf1840bc-1155-414a-bd1f-2e30695fadeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:03.832547 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 667/1319] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:56:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:56:03.889009 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ec0b0-69cb-4e77-a000-59b77126d697 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:03.889293 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ec0b0-69cb-4e77-a000-59b77126d697 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:03.889579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ec0b0-69cb-4e77-a000-59b77126d697 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:03.891003 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 653/1320] 10.222.0.76 () {54 vars in 964 bytes} [Thu May 26 17:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:56:03.897009 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30320c-ddb6-484f-8896-9bcee89a26cc None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:03.897377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30320c-ddb6-484f-8896-9bcee89a26cc None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:03.897696 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30320c-ddb6-484f-8896-9bcee89a26cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:03.919733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:56:03.976565 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-9b30320c-ddb6-484f-8896-9bcee89a26cc None None] MFA Rules not processed for user `8df66990a3734209a350c8b6768d259f`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:56:03.977766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9b30320c-ddb6-484f-8896-9bcee89a26cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:04.053942 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 668/1321] 10.222.0.76 () {58 vars in 1061 bytes} [Thu May 26 17:56:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:56:04.063318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ec0b0-69cb-4e77-a000-59b77126d697 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:56:04.064851 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-c35ec0b0-69cb-4e77-a000-59b77126d697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:04.173724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ec0b0-69cb-4e77-a000-59b77126d697 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:56:04.250288 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da648770-1ec6-4cf5-b488-70d22fc73ee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:56:04.250951 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da648770-1ec6-4cf5-b488-70d22fc73ee4 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:04.251241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da648770-1ec6-4cf5-b488-70d22fc73ee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:04.251519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da648770-1ec6-4cf5-b488-70d22fc73ee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:04.253202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-da648770-1ec6-4cf5-b488-70d22fc73ee4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:04.261078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:56:04.364214 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da648770-1ec6-4cf5-b488-70d22fc73ee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:56:04.365639 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da648770-1ec6-4cf5-b488-70d22fc73ee4 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:56:04.384170 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 654/1322] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:56:04 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:56:07.777834 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d4b92-c269-4791-8ad4-5cc0f7fe3c7a None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:07.778458 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d4b92-c269-4791-8ad4-5cc0f7fe3c7a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:07.778458 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d4b92-c269-4791-8ad4-5cc0f7fe3c7a None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:07.779813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 669/1323] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:56:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:56:07.785687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e857dc7-6148-454a-bccb-6d27a5b61f63 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:07.786209 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e857dc7-6148-454a-bccb-6d27a5b61f63 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:07.786312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e857dc7-6148-454a-bccb-6d27a5b61f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:07.859241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-0e857dc7-6148-454a-bccb-6d27a5b61f63 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:56:07.860369 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-0e857dc7-6148-454a-bccb-6d27a5b61f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:07.920610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:56:07.960634 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 655/1324] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:56:07.998785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d4b92-c269-4791-8ad4-5cc0f7fe3c7a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:56:08.000335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6b3d4b92-c269-4791-8ad4-5cc0f7fe3c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:08.108755 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d4b92-c269-4791-8ad4-5cc0f7fe3c7a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:56:08.122095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:56:08.191844 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fa39e-9e28-44f3-8062-4c85d8f5b4d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:56:08.192503 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fa39e-9e28-44f3-8062-4c85d8f5b4d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:08.192790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fa39e-9e28-44f3-8062-4c85d8f5b4d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:08.193069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fa39e-9e28-44f3-8062-4c85d8f5b4d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:08.194750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d35fa39e-9e28-44f3-8062-4c85d8f5b4d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:08.308774 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35fa39e-9e28-44f3-8062-4c85d8f5b4d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:56:08.310251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35fa39e-9e28-44f3-8062-4c85d8f5b4d1 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:56:08.330453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 670/1325] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:56:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:56:10.626171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b02659-1432-4a0c-ac6d-fbebbe0029f8 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:10.626755 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b02659-1432-4a0c-ac6d-fbebbe0029f8 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:10.626755 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b02659-1432-4a0c-ac6d-fbebbe0029f8 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:10.628169 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 656/1326] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:56:10.633945 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c368508-6e9a-440d-9488-8c401f8a0e88 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:10.634433 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c368508-6e9a-440d-9488-8c401f8a0e88 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:10.634554 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c368508-6e9a-440d-9488-8c401f8a0e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:10.683772 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:56:10.706093 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0c368508-6e9a-440d-9488-8c401f8a0e88 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:56:10.707344 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0c368508-6e9a-440d-9488-8c401f8a0e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:10.807327 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 671/1327] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:56:10.846640 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b02659-1432-4a0c-ac6d-fbebbe0029f8 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:56:10.848273 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-60b02659-1432-4a0c-ac6d-fbebbe0029f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:10.958051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b02659-1432-4a0c-ac6d-fbebbe0029f8 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:56:11.035631 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093206f-9e41-4b26-9d5f-762bf6fef445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:56:11.036292 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093206f-9e41-4b26-9d5f-762bf6fef445 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:11.036572 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093206f-9e41-4b26-9d5f-762bf6fef445 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:11.036849 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093206f-9e41-4b26-9d5f-762bf6fef445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:11.038554 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-3093206f-9e41-4b26-9d5f-762bf6fef445 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:11.057865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:56:11.159548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093206f-9e41-4b26-9d5f-762bf6fef445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:56:11.160979 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093206f-9e41-4b26-9d5f-762bf6fef445 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:56:11.180427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 657/1328] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:56:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:56:13.614975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c970dd7-9236-4c86-8fd9-231e282131d0 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:13.615520 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c970dd7-9236-4c86-8fd9-231e282131d0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:13.615520 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c970dd7-9236-4c86-8fd9-231e282131d0 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:13.616937 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 672/1329] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:56:13.623612 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb0a50-6c2b-469d-81b2-852d7940a00a None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:13.624387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb0a50-6c2b-469d-81b2-852d7940a00a None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:13.624578 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb0a50-6c2b-469d-81b2-852d7940a00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:13.710066 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-14bb0a50-6c2b-469d-81b2-852d7940a00a None None] MFA Rules not processed for user `95b4e2fbe1aa4fa5b7cebed5b01639cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:56:13.711722 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-14bb0a50-6c2b-469d-81b2-852d7940a00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:13.787427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:56:13.838417 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 658/1330] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:56:13 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:56:13.885144 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c970dd7-9236-4c86-8fd9-231e282131d0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:56:13.886775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7c970dd7-9236-4c86-8fd9-231e282131d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:14.006489 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c970dd7-9236-4c86-8fd9-231e282131d0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:56:14.028397 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:56:14.085907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c393b-a11d-423a-952a-2ef35dfc8e68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:56:14.086601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c393b-a11d-423a-952a-2ef35dfc8e68 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:14.086888 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c393b-a11d-423a-952a-2ef35dfc8e68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:14.087172 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c393b-a11d-423a-952a-2ef35dfc8e68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:14.088854 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1e7c393b-a11d-423a-952a-2ef35dfc8e68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:14.208567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c393b-a11d-423a-952a-2ef35dfc8e68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:56:14.210022 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c393b-a11d-423a-952a-2ef35dfc8e68 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:56:14.228637 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 673/1331] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:56:13 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:56:16.734869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c434e-8aae-4815-ae8d-dc5d0b423331 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:16.735399 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c434e-8aae-4815-ae8d-dc5d0b423331 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:16.735511 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c434e-8aae-4815-ae8d-dc5d0b423331 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:16.737698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 659/1332] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:56:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:56:16.743624 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b189a-918b-476b-bb5d-f2cc4a963f14 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:16.744136 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b189a-918b-476b-bb5d-f2cc4a963f14 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:16.744351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b189a-918b-476b-bb5d-f2cc4a963f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:16.820628 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:56:16.824439 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b94b189a-918b-476b-bb5d-f2cc4a963f14 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:56:16.826352 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b94b189a-918b-476b-bb5d-f2cc4a963f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:16.926491 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 674/1333] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:56:16.963020 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c434e-8aae-4815-ae8d-dc5d0b423331 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:56:16.965187 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-d59c434e-8aae-4815-ae8d-dc5d0b423331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:17.079740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c434e-8aae-4815-ae8d-dc5d0b423331 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:56:17.157363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca95310-8e3c-44d4-aabf-a09f3f43d5cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:56:17.158017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca95310-8e3c-44d4-aabf-a09f3f43d5cc service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:17.158322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca95310-8e3c-44d4-aabf-a09f3f43d5cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:17.158595 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca95310-8e3c-44d4-aabf-a09f3f43d5cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:17.160487 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-1ca95310-8e3c-44d4-aabf-a09f3f43d5cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:17.189095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:56:17.276370 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca95310-8e3c-44d4-aabf-a09f3f43d5cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:56:17.278435 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca95310-8e3c-44d4-aabf-a09f3f43d5cc service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:56:17.301554 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 660/1334] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:56:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:56:19.629807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d835d-bb3f-4b52-8bdc-edb9686e1eeb None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:19.630707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d835d-bb3f-4b52-8bdc-edb9686e1eeb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:19.630992 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d835d-bb3f-4b52-8bdc-edb9686e1eeb None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:19.632410 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 675/1335] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:56:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:56:19.643670 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b48fd0-d10f-436f-a2bb-3a7dee949b16 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:19.644203 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b48fd0-d10f-436f-a2bb-3a7dee949b16 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:19.644297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b48fd0-d10f-436f-a2bb-3a7dee949b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:19.713584 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-e5b48fd0-d10f-436f-a2bb-3a7dee949b16 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:56:19.715396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-e5b48fd0-d10f-436f-a2bb-3a7dee949b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:19.793203 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:56:19.817129 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 661/1336] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:56:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:56:19.856931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d835d-bb3f-4b52-8bdc-edb9686e1eeb None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:56:19.858434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-011d835d-bb3f-4b52-8bdc-edb9686e1eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:19.962454 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d835d-bb3f-4b52-8bdc-edb9686e1eeb None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:56:19.990728 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:56:20.039608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55aa23d-e9b7-4413-ab1d-ae95312bd41c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:56:20.040261 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55aa23d-e9b7-4413-ab1d-ae95312bd41c service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:20.040541 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55aa23d-e9b7-4413-ab1d-ae95312bd41c service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:20.040816 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55aa23d-e9b7-4413-ab1d-ae95312bd41c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:20.042959 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d55aa23d-e9b7-4413-ab1d-ae95312bd41c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:20.162331 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55aa23d-e9b7-4413-ab1d-ae95312bd41c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:56:20.163863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55aa23d-e9b7-4413-ab1d-ae95312bd41c service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:56:20.183592 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 676/1337] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:56:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:56:22.818626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc7a5c-be03-4dc9-a87e-5af3208743da None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:22.819162 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc7a5c-be03-4dc9-a87e-5af3208743da None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:22.819162 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc7a5c-be03-4dc9-a87e-5af3208743da None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:22.820541 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 662/1338] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:56:22.826828 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ad4c0-efe8-4d7f-86a2-1ad9c97d8488 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:22.827544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ad4c0-efe8-4d7f-86a2-1ad9c97d8488 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:22.827544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ad4c0-efe8-4d7f-86a2-1ad9c97d8488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:22.899298 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-042ad4c0-efe8-4d7f-86a2-1ad9c97d8488 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:56:22.900442 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-042ad4c0-efe8-4d7f-86a2-1ad9c97d8488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:22.909993 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:56:23.001078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 677/1339] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:56:22 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:56:23.037567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc7a5c-be03-4dc9-a87e-5af3208743da None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:56:23.039158 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-2afc7a5c-be03-4dc9-a87e-5af3208743da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:23.147353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc7a5c-be03-4dc9-a87e-5af3208743da None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:56:23.221439 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777b2f6-0645-4dce-a133-ec669a65cb0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:56:23.222009 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777b2f6-0645-4dce-a133-ec669a65cb0e service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:23.222330 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777b2f6-0645-4dce-a133-ec669a65cb0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:23.222613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777b2f6-0645-4dce-a133-ec669a65cb0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:23.224286 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-8777b2f6-0645-4dce-a133-ec669a65cb0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:23.260023 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:56:23.336534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777b2f6-0645-4dce-a133-ec669a65cb0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:56:23.337987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777b2f6-0645-4dce-a133-ec669a65cb0e service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:56:23.358284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 663/1340] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:56:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:56:27.808705 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aadb2e-e703-4632-89b0-b8eee0603448 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:27.809262 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aadb2e-e703-4632-89b0-b8eee0603448 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:27.809262 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aadb2e-e703-4632-89b0-b8eee0603448 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:27.810866 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 678/1341] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:56:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:56:27.816779 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8044ff8-7c89-4a22-853b-a99f62280eef None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:27.817333 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8044ff8-7c89-4a22-853b-a99f62280eef None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:27.817333 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8044ff8-7c89-4a22-853b-a99f62280eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:27.896754 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-a8044ff8-7c89-4a22-853b-a99f62280eef None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:56:27.897881 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-a8044ff8-7c89-4a22-853b-a99f62280eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:27.995296 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:56:28.009820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 664/1342] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:56:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:56:28.050063 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aadb2e-e703-4632-89b0-b8eee0603448 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:56:28.052657 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a0aadb2e-e703-4632-89b0-b8eee0603448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:28.160096 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aadb2e-e703-4632-89b0-b8eee0603448 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:56:28.163786 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c2b16-a3dc-4ec0-b20a-6b0316cbbafe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:56:28.164438 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c2b16-a3dc-4ec0-b20a-6b0316cbbafe service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:28.164723 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c2b16-a3dc-4ec0-b20a-6b0316cbbafe service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:28.164999 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c2b16-a3dc-4ec0-b20a-6b0316cbbafe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:28.174870 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c2b16-a3dc-4ec0-b20a-6b0316cbbafe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:56:28.176386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c2b16-a3dc-4ec0-b20a-6b0316cbbafe service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:56:28.187260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 679/1343] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:56:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:56:31.188591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647b99c-c5ee-41d2-b613-da8136c4a771 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:31.189129 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647b99c-c5ee-41d2-b613-da8136c4a771 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:31.189219 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647b99c-c5ee-41d2-b613-da8136c4a771 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:31.190923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 665/1344] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:56:31.196103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cb2f6-6709-435b-80b5-38b05b945aa1 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:31.196559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cb2f6-6709-435b-80b5-38b05b945aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:31.196654 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cb2f6-6709-435b-80b5-38b05b945aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:31.228814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2d9cb2f6-6709-435b-80b5-38b05b945aa1 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:56:31.231950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2d9cb2f6-6709-435b-80b5-38b05b945aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:31.275209 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:56:31.328725 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 680/1345] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:56:31 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:56:31.365735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647b99c-c5ee-41d2-b613-da8136c4a771 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:56:31.377317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647b99c-c5ee-41d2-b613-da8136c4a771 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:56:31.380865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f87769-63fa-46fa-80e4-6543a1a3801b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:56:31.381501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f87769-63fa-46fa-80e4-6543a1a3801b service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:31.381781 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f87769-63fa-46fa-80e4-6543a1a3801b service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:31.382054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f87769-63fa-46fa-80e4-6543a1a3801b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:31.393037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f87769-63fa-46fa-80e4-6543a1a3801b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:56:31.394502 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f87769-63fa-46fa-80e4-6543a1a3801b service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:56:31.405392 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 666/1346] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:56:31 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:56:36.018364 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfbb8b8-2499-467f-8c5d-01a4cd84894a None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:36.018901 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfbb8b8-2499-467f-8c5d-01a4cd84894a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:36.019089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfbb8b8-2499-467f-8c5d-01a4cd84894a None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:36.020574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 681/1347] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:56:36.026037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62b594-3dc1-46b0-89f0-3e3eb9f3015a None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:36.026670 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62b594-3dc1-46b0-89f0-3e3eb9f3015a None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:36.027705 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62b594-3dc1-46b0-89f0-3e3eb9f3015a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:36.048466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-8c62b594-3dc1-46b0-89f0-3e3eb9f3015a None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:56:36.049693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-8c62b594-3dc1-46b0-89f0-3e3eb9f3015a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:36.058308 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 667/1348] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:56:36.096756 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfbb8b8-2499-467f-8c5d-01a4cd84894a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:56:36.108535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfbb8b8-2499-467f-8c5d-01a4cd84894a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:56:36.112180 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299c31b-d875-4017-8af7-e90aa7584f1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:56:36.112915 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299c31b-d875-4017-8af7-e90aa7584f1d service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:36.113274 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299c31b-d875-4017-8af7-e90aa7584f1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:36.113614 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299c31b-d875-4017-8af7-e90aa7584f1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:36.124137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299c31b-d875-4017-8af7-e90aa7584f1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:56:36.125570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299c31b-d875-4017-8af7-e90aa7584f1d service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:56:36.138033 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 682/1349] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:56:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:56:41.637828 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedadf8-edec-42f8-a393-9f0fd4729ac2 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:41.638802 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedadf8-edec-42f8-a393-9f0fd4729ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:41.639248 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedadf8-edec-42f8-a393-9f0fd4729ac2 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:41.640955 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 668/1350] 10.222.0.76 () {54 vars in 921 bytes} [Thu May 26 17:56:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:56:41.646380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46309ae-6df1-43f4-8725-fecba73f4a70 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:41.647174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46309ae-6df1-43f4-8725-fecba73f4a70 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:41.647629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46309ae-6df1-43f4-8725-fecba73f4a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:41.728579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d46309ae-6df1-43f4-8725-fecba73f4a70 None None] MFA Rules not processed for user `596f126382e3440fb3cf7c357eef889f`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:56:41.730667 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d46309ae-6df1-43f4-8725-fecba73f4a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:41.828284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 683/1351] 10.222.0.76 () {58 vars in 1018 bytes} [Thu May 26 17:56:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:56:46.042590 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=13 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:56:46.247693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9102c22-2e12-4992-a8db-30f0babfe870 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:46.248053 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9102c22-2e12-4992-a8db-30f0babfe870 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:46.248255 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9102c22-2e12-4992-a8db-30f0babfe870 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:46.249794 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 669/1352] 10.222.0.76 () {54 vars in 921 bytes} [Thu May 26 17:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:56:46.255070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4c367-f06f-433c-a4e7-8769ba8c2957 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:46.256016 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4c367-f06f-433c-a4e7-8769ba8c2957 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:46.256523 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4c367-f06f-433c-a4e7-8769ba8c2957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:46.278126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-78b4c367-f06f-433c-a4e7-8769ba8c2957 None None] MFA Rules not processed for user `596f126382e3440fb3cf7c357eef889f`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:56:46.279984 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-78b4c367-f06f-433c-a4e7-8769ba8c2957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:46.290732 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 684/1353] 10.222.0.76 () {58 vars in 1018 bytes} [Thu May 26 17:56:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:56:47.871209 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b63c5-3fb5-4237-a692-d04a773326a1 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:47.871806 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b63c5-3fb5-4237-a692-d04a773326a1 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:47.871806 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b63c5-3fb5-4237-a692-d04a773326a1 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:47.873530 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 670/1354] 10.222.0.76 () {54 vars in 921 bytes} [Thu May 26 17:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:56:47.879903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169e89a-287b-4943-913f-be6765e7aeb3 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:47.880342 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169e89a-287b-4943-913f-be6765e7aeb3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:47.880448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169e89a-287b-4943-913f-be6765e7aeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:47.898486 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2169e89a-287b-4943-913f-be6765e7aeb3 None None] MFA Rules not processed for user `596f126382e3440fb3cf7c357eef889f`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:56:47.899682 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2169e89a-287b-4943-913f-be6765e7aeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:47.909614 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 685/1355] 10.222.0.76 () {58 vars in 1018 bytes} [Thu May 26 17:56:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:56:51.877619 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d53502-629f-4322-99cd-80b7439a9c06 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:51.878159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d53502-629f-4322-99cd-80b7439a9c06 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:51.878159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d53502-629f-4322-99cd-80b7439a9c06 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:51.879482 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 671/1356] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:56:51.884726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d9608-cb50-49c9-ae47-e4b8142cf577 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:51.885709 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d9608-cb50-49c9-ae47-e4b8142cf577 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:51.885821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d9608-cb50-49c9-ae47-e4b8142cf577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:51.905524 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-267d9608-cb50-49c9-ae47-e4b8142cf577 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:56:51.906779 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-267d9608-cb50-49c9-ae47-e4b8142cf577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:51.915652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 686/1357] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:56:51.973533 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942651d-b21b-4879-8b02-1238e7e45abc None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:51.973769 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942651d-b21b-4879-8b02-1238e7e45abc None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:51.974083 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942651d-b21b-4879-8b02-1238e7e45abc None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:51.975712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 672/1358] 10.222.0.76 () {54 vars in 961 bytes} [Thu May 26 17:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:56:51.996228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f41bab3-3c65-468a-a2a2-f8b604ed33d8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:51.997187 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f41bab3-3c65-468a-a2a2-f8b604ed33d8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:51.997732 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f41bab3-3c65-468a-a2a2-f8b604ed33d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:52.048747 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:56:52.057364 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1f41bab3-3c65-468a-a2a2-f8b604ed33d8 None None] MFA Rules not processed for user `1899c317083d40f5b490cd9eb63fe521`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:56:52.058760 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1f41bab3-3c65-468a-a2a2-f8b604ed33d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:52.160454 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 687/1359] 10.222.0.76 () {58 vars in 1058 bytes} [Thu May 26 17:56:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:56:52.169576 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942651d-b21b-4879-8b02-1238e7e45abc None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:56:52.186967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942651d-b21b-4879-8b02-1238e7e45abc None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:56:52.193154 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b9b55-a8c3-4ff5-b18b-9e14bb603620 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:56:52.194234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b9b55-a8c3-4ff5-b18b-9e14bb603620 service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:52.194611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b9b55-a8c3-4ff5-b18b-9e14bb603620 service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:52.194959 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b9b55-a8c3-4ff5-b18b-9e14bb603620 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:52.205511 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b9b55-a8c3-4ff5-b18b-9e14bb603620 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:56:52.207462 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b9b55-a8c3-4ff5-b18b-9e14bb603620 service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:56:52.220176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 673/1360] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:56:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 17:56:54.743564 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d273b-7a3e-40a1-9030-5773a960b570 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:54.744119 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d273b-7a3e-40a1-9030-5773a960b570 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:54.744119 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d273b-7a3e-40a1-9030-5773a960b570 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:54.745598 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 688/1361] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:56:54.751442 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129c536-da78-4cb7-8cb7-11f302e032d8 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:54.751977 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129c536-da78-4cb7-8cb7-11f302e032d8 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:54.751977 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129c536-da78-4cb7-8cb7-11f302e032d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:54.770459 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-5129c536-da78-4cb7-8cb7-11f302e032d8 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:56:54.771652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-5129c536-da78-4cb7-8cb7-11f302e032d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:54.780387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 674/1362] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:56:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:56:54.826405 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d273b-7a3e-40a1-9030-5773a960b570 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:56:54.842968 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d273b-7a3e-40a1-9030-5773a960b570 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:56:54.848539 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e6d345-a011-4005-8eaf-6d7f9ad99919 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:56:54.849196 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6d345-a011-4005-8eaf-6d7f9ad99919 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:54.849487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6d345-a011-4005-8eaf-6d7f9ad99919 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:54.849767 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6d345-a011-4005-8eaf-6d7f9ad99919 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:54.859259 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e6d345-a011-4005-8eaf-6d7f9ad99919 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:56:54.860930 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e6d345-a011-4005-8eaf-6d7f9ad99919 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:56:54.871791 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 689/1363] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:56:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:56:57.517011 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab323c-cc67-48fa-942c-6830764176b5 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:57.518305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab323c-cc67-48fa-942c-6830764176b5 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:57.518603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab323c-cc67-48fa-942c-6830764176b5 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:57.520053 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 675/1364] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:56:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:56:57.525467 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862cf31f-7a2e-49e0-9b90-b0f5e7fe9eea None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:57.526390 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862cf31f-7a2e-49e0-9b90-b0f5e7fe9eea None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:57.526671 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862cf31f-7a2e-49e0-9b90-b0f5e7fe9eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:57.545701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-862cf31f-7a2e-49e0-9b90-b0f5e7fe9eea None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:56:57.548443 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-862cf31f-7a2e-49e0-9b90-b0f5e7fe9eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:57.568624 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 690/1365] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:56:57.624209 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9df18-6bc5-4ccb-9553-cc0acd1ee0e9 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:57.624491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9df18-6bc5-4ccb-9553-cc0acd1ee0e9 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:57.624952 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9df18-6bc5-4ccb-9553-cc0acd1ee0e9 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:57.626332 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 676/1366] 10.222.0.76 () {54 vars in 961 bytes} [Thu May 26 17:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:56:57.632264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d24fa1-18e1-4324-ae80-bf5d6c8d0b21 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:57.632656 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d24fa1-18e1-4324-ae80-bf5d6c8d0b21 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:57.632988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d24fa1-18e1-4324-ae80-bf5d6c8d0b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:57.653612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e1d24fa1-18e1-4324-ae80-bf5d6c8d0b21 None None] MFA Rules not processed for user `1899c317083d40f5b490cd9eb63fe521`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:56:57.654882 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e1d24fa1-18e1-4324-ae80-bf5d6c8d0b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:56:57.664653 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 691/1367] 10.222.0.76 () {58 vars in 1058 bytes} [Thu May 26 17:56:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:56:57.672680 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9df18-6bc5-4ccb-9553-cc0acd1ee0e9 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:56:57.688558 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9df18-6bc5-4ccb-9553-cc0acd1ee0e9 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:56:57.693996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70ac5d-260b-4d43-b646-311e64b86b1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:56:57.694689 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70ac5d-260b-4d43-b646-311e64b86b1e service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:56:57.694979 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70ac5d-260b-4d43-b646-311e64b86b1e service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:56:57.695259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70ac5d-260b-4d43-b646-311e64b86b1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:56:57.705068 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70ac5d-260b-4d43-b646-311e64b86b1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:56:57.706850 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70ac5d-260b-4d43-b646-311e64b86b1e service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:56:57.717708 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 677/1368] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:56:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:00.064177 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1eb53-a4b2-4628-bcdb-903c2e32faa5 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:00.064718 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1eb53-a4b2-4628-bcdb-903c2e32faa5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:00.064843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1eb53-a4b2-4628-bcdb-903c2e32faa5 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:00.066378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 692/1369] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:00.076265 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13dcbd-6d76-4dd2-a74c-b2337e882343 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:00.077087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13dcbd-6d76-4dd2-a74c-b2337e882343 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:00.077087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13dcbd-6d76-4dd2-a74c-b2337e882343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:00.100536 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-3e13dcbd-6d76-4dd2-a74c-b2337e882343 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:00.101765 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-3e13dcbd-6d76-4dd2-a74c-b2337e882343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:00.110603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 678/1370] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:00.159383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1eb53-a4b2-4628-bcdb-903c2e32faa5 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:00.174963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1eb53-a4b2-4628-bcdb-903c2e32faa5 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:00.180415 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce64e0-3733-43e4-a490-c83fb2559ee6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:00.181296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce64e0-3733-43e4-a490-c83fb2559ee6 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:00.181631 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce64e0-3733-43e4-a490-c83fb2559ee6 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:00.181906 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce64e0-3733-43e4-a490-c83fb2559ee6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:00.192020 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ce64e0-3733-43e4-a490-c83fb2559ee6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:00.193779 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ce64e0-3733-43e4-a490-c83fb2559ee6 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:00.206009 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 693/1371] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:57:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:00.395210 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53256ef-27bc-47e2-a9e6-0cc5543d2967 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:00.395398 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53256ef-27bc-47e2-a9e6-0cc5543d2967 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:00.395897 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53256ef-27bc-47e2-a9e6-0cc5543d2967 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:00.398013 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 679/1372] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:57:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:00.403370 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a57a5f-9315-4e04-a921-da0ea249f414 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:00.403583 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a57a5f-9315-4e04-a921-da0ea249f414 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:00.403848 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a57a5f-9315-4e04-a921-da0ea249f414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:00.424152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d7a57a5f-9315-4e04-a921-da0ea249f414 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:00.425407 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d7a57a5f-9315-4e04-a921-da0ea249f414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:00.438240 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 694/1373] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:00.722317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708a0b4-8133-4ca0-ab23-2b6a79454637 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:00.722841 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708a0b4-8133-4ca0-ab23-2b6a79454637 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:00.723127 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708a0b4-8133-4ca0-ab23-2b6a79454637 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:00.724612 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 680/1374] 10.222.0.76 () {54 vars in 963 bytes} [Thu May 26 17:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:00.731961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47715c4-56ad-4dff-80c3-12322cc93f29 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:00.731961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47715c4-56ad-4dff-80c3-12322cc93f29 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:00.731961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47715c4-56ad-4dff-80c3-12322cc93f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:00.786999 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a47715c4-56ad-4dff-80c3-12322cc93f29 None None] MFA Rules not processed for user `76dbf8f3f1b44baca2d2aa90fe606a30`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:00.788217 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a47715c4-56ad-4dff-80c3-12322cc93f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:00.863342 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 695/1375] 10.222.0.76 () {58 vars in 1060 bytes} [Thu May 26 17:57:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:57:00.871561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708a0b4-8133-4ca0-ab23-2b6a79454637 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:00.883577 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708a0b4-8133-4ca0-ab23-2b6a79454637 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:00.889189 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeea611-ad51-437c-b923-5d844055b10f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:00.889976 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeea611-ad51-437c-b923-5d844055b10f service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:00.890374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeea611-ad51-437c-b923-5d844055b10f service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:00.890736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeea611-ad51-437c-b923-5d844055b10f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:00.901556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeea611-ad51-437c-b923-5d844055b10f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:00.903214 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeea611-ad51-437c-b923-5d844055b10f service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:00.914386 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 681/1376] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:57:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:02.486638 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015ba32-085f-43f1-9f47-5bd991beee6a None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:02.487219 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015ba32-085f-43f1-9f47-5bd991beee6a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:02.487219 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015ba32-085f-43f1-9f47-5bd991beee6a None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:02.488655 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 696/1377] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:02.494801 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c13764-059c-43be-9707-d04f41e162be None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:02.495382 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c13764-059c-43be-9707-d04f41e162be None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:02.495382 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c13764-059c-43be-9707-d04f41e162be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:02.515312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-00c13764-059c-43be-9707-d04f41e162be None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:02.516559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-00c13764-059c-43be-9707-d04f41e162be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:02.524696 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 682/1378] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:02.573090 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5015ba32-085f-43f1-9f47-5bd991beee6a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:02.588793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5015ba32-085f-43f1-9f47-5bd991beee6a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:02.594402 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6dc61-1f31-494d-afa4-6d2e211e7232 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:02.595078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6dc61-1f31-494d-afa4-6d2e211e7232 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:02.595363 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6dc61-1f31-494d-afa4-6d2e211e7232 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:02.595645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6dc61-1f31-494d-afa4-6d2e211e7232 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:02.606538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd6dc61-1f31-494d-afa4-6d2e211e7232 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:02.607657 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd6dc61-1f31-494d-afa4-6d2e211e7232 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:02.612361 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:57:02.630607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 697/1379] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:57:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:04.957625 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4c9ea-e19b-4cc4-826c-2015e1d86f34 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:04.958182 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4c9ea-e19b-4cc4-826c-2015e1d86f34 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:04.958182 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4c9ea-e19b-4cc4-826c-2015e1d86f34 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:04.959485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 683/1380] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:04.965716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92c634-6644-4fef-81b3-6a935f28560e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:04.966184 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92c634-6644-4fef-81b3-6a935f28560e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:04.966323 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92c634-6644-4fef-81b3-6a935f28560e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:04.984833 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-9b92c634-6644-4fef-81b3-6a935f28560e None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:04.986052 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9b92c634-6644-4fef-81b3-6a935f28560e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:04.995070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 698/1381] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:57:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:05.039790 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4c9ea-e19b-4cc4-826c-2015e1d86f34 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:05.054719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4c9ea-e19b-4cc4-826c-2015e1d86f34 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:05.061171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a420c-5f3d-44c6-a931-f2df9cd463be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:05.061851 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a420c-5f3d-44c6-a931-f2df9cd463be service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:05.062142 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a420c-5f3d-44c6-a931-f2df9cd463be service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:05.062470 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a420c-5f3d-44c6-a931-f2df9cd463be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:05.071559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a420c-5f3d-44c6-a931-f2df9cd463be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:05.073748 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a420c-5f3d-44c6-a931-f2df9cd463be service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:05.084158 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 684/1382] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:57:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:07.447802 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a161c-3549-4e64-acdf-f5675929392f None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:07.448435 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a161c-3549-4e64-acdf-f5675929392f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:07.448435 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a161c-3549-4e64-acdf-f5675929392f None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:07.449727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 699/1383] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:57:07.455271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a3593-4e58-427a-a0d8-4f4e90396b20 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:07.455698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a3593-4e58-427a-a0d8-4f4e90396b20 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:07.455809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a3593-4e58-427a-a0d8-4f4e90396b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:07.474587 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-8c2a3593-4e58-427a-a0d8-4f4e90396b20 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:07.475789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-8c2a3593-4e58-427a-a0d8-4f4e90396b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:07.485483 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 685/1384] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:07.529537 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a161c-3549-4e64-acdf-f5675929392f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:07.546852 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a161c-3549-4e64-acdf-f5675929392f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:07.552927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f95908-ca14-44cc-8ea9-f612c55fdfbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:07.553616 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f95908-ca14-44cc-8ea9-f612c55fdfbd service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:07.553892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f95908-ca14-44cc-8ea9-f612c55fdfbd service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:07.554172 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f95908-ca14-44cc-8ea9-f612c55fdfbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:07.564295 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f95908-ca14-44cc-8ea9-f612c55fdfbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:07.565923 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f95908-ca14-44cc-8ea9-f612c55fdfbd service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:07.578382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 700/1385] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:57:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:10.123530 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80880c0d-d56d-41f8-bc1c-78643543f072 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:10.124072 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80880c0d-d56d-41f8-bc1c-78643543f072 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:10.124354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80880c0d-d56d-41f8-bc1c-78643543f072 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:10.126117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 686/1386] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:57:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:10.132916 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f74b8-3be0-4b93-b83b-2b15ae3e7127 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:10.133395 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f74b8-3be0-4b93-b83b-2b15ae3e7127 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:10.133491 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f74b8-3be0-4b93-b83b-2b15ae3e7127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:10.155809 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-707f74b8-3be0-4b93-b83b-2b15ae3e7127 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:10.157040 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-707f74b8-3be0-4b93-b83b-2b15ae3e7127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:10.165394 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 701/1387] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:57:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:10.220117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80880c0d-d56d-41f8-bc1c-78643543f072 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:10.235746 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80880c0d-d56d-41f8-bc1c-78643543f072 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:10.241750 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab9953-d51d-44e3-8e86-4d37d23f3c57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:10.242463 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab9953-d51d-44e3-8e86-4d37d23f3c57 service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:10.242768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab9953-d51d-44e3-8e86-4d37d23f3c57 service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:10.243047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab9953-d51d-44e3-8e86-4d37d23f3c57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:10.252674 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab9953-d51d-44e3-8e86-4d37d23f3c57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:10.254384 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab9953-d51d-44e3-8e86-4d37d23f3c57 service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:10.265276 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 687/1388] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:57:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:12.536512 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d3b99-b4a8-41a6-8d94-14538e95b771 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:12.537068 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d3b99-b4a8-41a6-8d94-14538e95b771 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:12.537309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d3b99-b4a8-41a6-8d94-14538e95b771 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:12.539225 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 702/1389] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:57:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:12.545523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafede5-9d8d-45fc-8d8c-8a11a89512b8 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:12.546084 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafede5-9d8d-45fc-8d8c-8a11a89512b8 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:12.546084 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafede5-9d8d-45fc-8d8c-8a11a89512b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:12.564242 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-acafede5-9d8d-45fc-8d8c-8a11a89512b8 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:12.565456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-acafede5-9d8d-45fc-8d8c-8a11a89512b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:12.573418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 688/1390] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:57:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:12.618741 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d3b99-b4a8-41a6-8d94-14538e95b771 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:12.634242 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d3b99-b4a8-41a6-8d94-14538e95b771 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:12.640623 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048be64-154b-4e34-8507-ea1669002bdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:12.641301 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048be64-154b-4e34-8507-ea1669002bdf service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:12.641662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048be64-154b-4e34-8507-ea1669002bdf service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:12.641880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048be64-154b-4e34-8507-ea1669002bdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:12.652698 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048be64-154b-4e34-8507-ea1669002bdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:12.654338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048be64-154b-4e34-8507-ea1669002bdf service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:12.664429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 703/1391] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:57:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:15.447355 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49df50d-6274-4a19-96eb-f29a40fd80a4 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:15.447969 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49df50d-6274-4a19-96eb-f29a40fd80a4 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:15.447969 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49df50d-6274-4a19-96eb-f29a40fd80a4 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:15.449204 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 689/1392] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:15.454911 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b57c6-26a9-4341-bdec-8b6715241bb9 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:15.455405 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b57c6-26a9-4341-bdec-8b6715241bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:15.455510 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b57c6-26a9-4341-bdec-8b6715241bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:15.474464 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-346b57c6-26a9-4341-bdec-8b6715241bb9 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:15.475747 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-346b57c6-26a9-4341-bdec-8b6715241bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:15.484684 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 704/1393] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:15.530366 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49df50d-6274-4a19-96eb-f29a40fd80a4 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:15.546424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49df50d-6274-4a19-96eb-f29a40fd80a4 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:15.552085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b2d07-3b96-497a-9e48-c1ea8be11acd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:15.552737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b2d07-3b96-497a-9e48-c1ea8be11acd service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:15.553027 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b2d07-3b96-497a-9e48-c1ea8be11acd service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:15.553308 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b2d07-3b96-497a-9e48-c1ea8be11acd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:15.562692 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b2d07-3b96-497a-9e48-c1ea8be11acd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:15.564338 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b2d07-3b96-497a-9e48-c1ea8be11acd service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:15.578351 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 690/1394] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:57:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:17.043439 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f7d5f-820a-4cbf-a36a-cddcf37835eb None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:17.044039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f7d5f-820a-4cbf-a36a-cddcf37835eb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:17.044039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f7d5f-820a-4cbf-a36a-cddcf37835eb None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:17.045440 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 705/1395] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:17.051518 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3026359-f46b-438a-970d-4814d22d1dd7 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:17.051958 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3026359-f46b-438a-970d-4814d22d1dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:17.052183 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3026359-f46b-438a-970d-4814d22d1dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:17.059177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=52 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:57:17.071754 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-e3026359-f46b-438a-970d-4814d22d1dd7 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:17.073676 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-e3026359-f46b-438a-970d-4814d22d1dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:17.081579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 691/1396] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:17.138303 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b310a7-5f59-4f6b-b82a-3af310ffce53 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:17.138599 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b310a7-5f59-4f6b-b82a-3af310ffce53 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:17.139123 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b310a7-5f59-4f6b-b82a-3af310ffce53 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:17.140708 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 706/1397] 10.222.0.76 () {54 vars in 962 bytes} [Thu May 26 17:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:17.146231 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925004c-1a45-474f-83a9-c2d2b52345da None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:17.146527 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925004c-1a45-474f-83a9-c2d2b52345da None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:17.146812 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925004c-1a45-474f-83a9-c2d2b52345da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:17.198243 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-2925004c-1a45-474f-83a9-c2d2b52345da None None] MFA Rules not processed for user `52f4c9fc949c4a2c951b755234848cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:17.199307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-2925004c-1a45-474f-83a9-c2d2b52345da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:17.262210 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 692/1398] 10.222.0.76 () {58 vars in 1059 bytes} [Thu May 26 17:57:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:17.269759 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b310a7-5f59-4f6b-b82a-3af310ffce53 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:17.280706 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b310a7-5f59-4f6b-b82a-3af310ffce53 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:17.284512 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743473b9-11a4-4ef5-a65c-ea6cccebfae0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f4c9fc949c4a2c951b755234848cc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:17.285159 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743473b9-11a4-4ef5-a65c-ea6cccebfae0 service glance] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:17.285449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743473b9-11a4-4ef5-a65c-ea6cccebfae0 service glance] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:17.285729 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743473b9-11a4-4ef5-a65c-ea6cccebfae0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:17.296375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743473b9-11a4-4ef5-a65c-ea6cccebfae0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:17.297754 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743473b9-11a4-4ef5-a65c-ea6cccebfae0 service glance] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:17.313496 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 707/1399] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:57:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:17.860765 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83110a5e-87f3-447e-9b3a-dbef3f1dddfb None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:17.861066 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83110a5e-87f3-447e-9b3a-dbef3f1dddfb None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:17.861348 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83110a5e-87f3-447e-9b3a-dbef3f1dddfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:17.886115 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3a63f-c747-4c96-91d9-df7688802f6d None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:17.886540 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3a63f-c747-4c96-91d9-df7688802f6d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:17.886900 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3a63f-c747-4c96-91d9-df7688802f6d None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:17.888369 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 708/1400] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:57:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:57:17.893659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cd799-69d2-4bc5-b796-ccd0161bfbed None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:17.894017 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cd799-69d2-4bc5-b796-ccd0161bfbed None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:17.894394 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cd799-69d2-4bc5-b796-ccd0161bfbed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:17.911481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-83110a5e-87f3-447e-9b3a-dbef3f1dddfb None None] MFA Rules not processed for user `3e06d1b0e1a9472db764f4be7e42fdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:17.912730 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-83110a5e-87f3-447e-9b3a-dbef3f1dddfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:17.913278 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-334cd799-69d2-4bc5-b796-ccd0161bfbed None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:17.914502 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-334cd799-69d2-4bc5-b796-ccd0161bfbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:17.923196 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 709/1401] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:57:17.967220 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cd799-69d2-4bc5-b796-ccd0161bfbed None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:17.979913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 693/1402] 10.222.0.76 () {58 vars in 1009 bytes} [Thu May 26 17:57:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:17.980557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cd799-69d2-4bc5-b796-ccd0161bfbed None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:17.986214 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac425e-3df3-4183-a76f-d774abc72185 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:17.986874 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac425e-3df3-4183-a76f-d774abc72185 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:17.987164 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac425e-3df3-4183-a76f-d774abc72185 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:17.987440 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac425e-3df3-4183-a76f-d774abc72185 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:17.996615 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac425e-3df3-4183-a76f-d774abc72185 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:17.998394 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac425e-3df3-4183-a76f-d774abc72185 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:18.008984 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 710/1403] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:57:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 17:57:18.023475 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66178ef-7d1e-470a-9cbb-00db7f91aea4 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:18.024010 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66178ef-7d1e-470a-9cbb-00db7f91aea4 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:18.024428 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66178ef-7d1e-470a-9cbb-00db7f91aea4 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:18.027202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 694/1404] 10.222.0.76 () {54 vars in 942 bytes} [Thu May 26 17:57:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:18.035157 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c15fd-a8e2-4176-95a2-d3aa073857d4 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:18.035457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c15fd-a8e2-4176-95a2-d3aa073857d4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:18.035923 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c15fd-a8e2-4176-95a2-d3aa073857d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:18.093505 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ea2c15fd-a8e2-4176-95a2-d3aa073857d4 None None] MFA Rules not processed for user `9ca774e864ca44cfb187e38bc8a8f6ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:18.095024 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ea2c15fd-a8e2-4176-95a2-d3aa073857d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:18.181459 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 711/1405] 10.222.0.76 () {58 vars in 1039 bytes} [Thu May 26 17:57:18 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:18.191705 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66178ef-7d1e-470a-9cbb-00db7f91aea4 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:18.206369 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66178ef-7d1e-470a-9cbb-00db7f91aea4 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:18.211776 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6512af-82d8-40b0-a10f-bb3c70cdbad8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ca774e864ca44cfb187e38bc8a8f6ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:18.213170 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6512af-82d8-40b0-a10f-bb3c70cdbad8 service swift] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:18.213525 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6512af-82d8-40b0-a10f-bb3c70cdbad8 service swift] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:18.213867 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6512af-82d8-40b0-a10f-bb3c70cdbad8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:18.223504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6512af-82d8-40b0-a10f-bb3c70cdbad8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:18.225239 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6512af-82d8-40b0-a10f-bb3c70cdbad8 service swift] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:18.235073 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 695/1406] 10.222.0.76 () {60 vars in 1384 bytes} [Thu May 26 17:57:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 17:57:20.356173 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18ab85-43a6-48ee-b558-8f52bc9fb4f2 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:20.356700 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18ab85-43a6-48ee-b558-8f52bc9fb4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:20.356700 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18ab85-43a6-48ee-b558-8f52bc9fb4f2 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:20.358117 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 712/1407] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:57:20.363608 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca2845-6118-4635-9067-e9b71d902bc2 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:20.364044 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca2845-6118-4635-9067-e9b71d902bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:20.364221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca2845-6118-4635-9067-e9b71d902bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:20.386914 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-9fca2845-6118-4635-9067-e9b71d902bc2 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:20.388109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-9fca2845-6118-4635-9067-e9b71d902bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:20.396880 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 696/1408] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:57:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:20.444623 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18ab85-43a6-48ee-b558-8f52bc9fb4f2 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:20.463130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18ab85-43a6-48ee-b558-8f52bc9fb4f2 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:20.469510 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2841c-7d2d-4d2f-a01c-812fe7620791 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:20.470383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2841c-7d2d-4d2f-a01c-812fe7620791 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:20.470674 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2841c-7d2d-4d2f-a01c-812fe7620791 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:20.470957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2841c-7d2d-4d2f-a01c-812fe7620791 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:20.481395 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f2841c-7d2d-4d2f-a01c-812fe7620791 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:20.483111 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f2841c-7d2d-4d2f-a01c-812fe7620791 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:20.496866 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 713/1409] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:57:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:30.381991 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:57:30.489352 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=87 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:57:42.102344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def37f39-87c9-4beb-9225-179046122fc9 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:42.103086 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def37f39-87c9-4beb-9225-179046122fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:42.103086 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def37f39-87c9-4beb-9225-179046122fc9 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:42.104358 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 697/1410] 10.222.0.76 () {54 vars in 918 bytes} [Thu May 26 17:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:42.109607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101d0c3-7eae-410e-b5a5-8aa90a6fc946 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:42.110074 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101d0c3-7eae-410e-b5a5-8aa90a6fc946 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:42.110183 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101d0c3-7eae-410e-b5a5-8aa90a6fc946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:42.128712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b101d0c3-7eae-410e-b5a5-8aa90a6fc946 None None] MFA Rules not processed for user `596f126382e3440fb3cf7c357eef889f`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:42.129918 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b101d0c3-7eae-410e-b5a5-8aa90a6fc946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:42.140024 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 714/1411] 10.222.0.76 () {58 vars in 1015 bytes} [Thu May 26 17:57:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:42.155564 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756a52b-147b-43e2-9c16-7c45c3e0318c None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:42.155859 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756a52b-147b-43e2-9c16-7c45c3e0318c None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:42.156145 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756a52b-147b-43e2-9c16-7c45c3e0318c None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:42.157541 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 698/1412] 10.222.0.76 () {54 vars in 918 bytes} [Thu May 26 17:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:42.162614 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce6c0b-4b4e-4c74-9abf-93471ff2682b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:42.162909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce6c0b-4b4e-4c74-9abf-93471ff2682b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:42.163191 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce6c0b-4b4e-4c74-9abf-93471ff2682b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:42.181244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ebce6c0b-4b4e-4c74-9abf-93471ff2682b None None] MFA Rules not processed for user `1899c317083d40f5b490cd9eb63fe521`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:42.182476 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ebce6c0b-4b4e-4c74-9abf-93471ff2682b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:42.191664 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 715/1413] 10.222.0.76 () {58 vars in 1015 bytes} [Thu May 26 17:57:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:44.938735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9557e5d-c8af-407e-8437-e61c82710058 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:44.939291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9557e5d-c8af-407e-8437-e61c82710058 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:44.939291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9557e5d-c8af-407e-8437-e61c82710058 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:44.940702 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:44.946575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c1545-4f6b-4913-abd9-2368f37be2d0 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:44.947055 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c1545-4f6b-4913-abd9-2368f37be2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:44.947220 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c1545-4f6b-4913-abd9-2368f37be2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:44.965303 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1f1c1545-4f6b-4913-abd9-2368f37be2d0 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:44.966545 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1f1c1545-4f6b-4913-abd9-2368f37be2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:44.974319 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:57:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:45.021614 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9557e5d-c8af-407e-8437-e61c82710058 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:45.037833 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9557e5d-c8af-407e-8437-e61c82710058 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:45.044349 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540056c-36c0-46e9-9ac4-ef6f54fa5a5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:45.045027 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540056c-36c0-46e9-9ac4-ef6f54fa5a5a service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:45.045286 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540056c-36c0-46e9-9ac4-ef6f54fa5a5a service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:45.045566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540056c-36c0-46e9-9ac4-ef6f54fa5a5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:45.056326 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540056c-36c0-46e9-9ac4-ef6f54fa5a5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:45.057927 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540056c-36c0-46e9-9ac4-ef6f54fa5a5a service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:45.067727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 700/1416] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:57:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:46.062909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5b0b6-7921-4687-a55b-b3194ce6a5ed None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:46.063459 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5b0b6-7921-4687-a55b-b3194ce6a5ed None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:46.063459 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5b0b6-7921-4687-a55b-b3194ce6a5ed None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:46.064842 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 17:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:46.070064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a220119-2684-4fac-95e5-3a05707d0bfc None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:46.070589 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a220119-2684-4fac-95e5-3a05707d0bfc None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:46.070686 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a220119-2684-4fac-95e5-3a05707d0bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:46.090765 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-6a220119-2684-4fac-95e5-3a05707d0bfc None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:46.091982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-6a220119-2684-4fac-95e5-3a05707d0bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:46.100588 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 17:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:46.284784 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5b0b6-7921-4687-a55b-b3194ce6a5ed None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:46.300081 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5b0b6-7921-4687-a55b-b3194ce6a5ed None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:46.305972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1392dd-1144-4fe6-9371-644ca0a09cf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:46.306264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1392dd-1144-4fe6-9371-644ca0a09cf2 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:46.306557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1392dd-1144-4fe6-9371-644ca0a09cf2 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:46.306840 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1392dd-1144-4fe6-9371-644ca0a09cf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:46.315462 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1392dd-1144-4fe6-9371-644ca0a09cf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:46.316995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1392dd-1144-4fe6-9371-644ca0a09cf2 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:46.326607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 718/1419] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:57:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:47.271190 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583fafe-131f-4bbe-9054-44d65d0ae2b8 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:47.271747 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583fafe-131f-4bbe-9054-44d65d0ae2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:47.271747 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583fafe-131f-4bbe-9054-44d65d0ae2b8 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:47.273143 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 17:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:47.278400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03229c56-d792-4dea-a1cd-0fc42b8b1efc None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:47.278820 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03229c56-d792-4dea-a1cd-0fc42b8b1efc None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:47.278961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03229c56-d792-4dea-a1cd-0fc42b8b1efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:47.297308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-03229c56-d792-4dea-a1cd-0fc42b8b1efc None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:47.298613 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-03229c56-d792-4dea-a1cd-0fc42b8b1efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:47.307000 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 17:57:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:47.462924 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583fafe-131f-4bbe-9054-44d65d0ae2b8 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:47.477942 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583fafe-131f-4bbe-9054-44d65d0ae2b8 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:47.483846 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cdbac2-603a-47c0-ba4c-ee7db82c6550 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:47.484593 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdbac2-603a-47c0-ba4c-ee7db82c6550 service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:47.484910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdbac2-603a-47c0-ba4c-ee7db82c6550 service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:47.485167 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdbac2-603a-47c0-ba4c-ee7db82c6550 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:47.495583 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cdbac2-603a-47c0-ba4c-ee7db82c6550 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:47.497205 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cdbac2-603a-47c0-ba4c-ee7db82c6550 service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:47.507993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 703/1422] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:57:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:48.450992 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb714b-49bd-4859-b021-b32c2e831a47 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:48.451528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb714b-49bd-4859-b021-b32c2e831a47 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:48.451528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb714b-49bd-4859-b021-b32c2e831a47 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:48.452907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 17:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:48.457999 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8930865-4d60-44f4-b0e5-e0ca7faa67d2 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:48.458424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8930865-4d60-44f4-b0e5-e0ca7faa67d2 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:48.458601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8930865-4d60-44f4-b0e5-e0ca7faa67d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:48.477202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-d8930865-4d60-44f4-b0e5-e0ca7faa67d2 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:48.478607 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-d8930865-4d60-44f4-b0e5-e0ca7faa67d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:48.486581 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 17:57:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:48.640506 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb714b-49bd-4859-b021-b32c2e831a47 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:48.657306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb714b-49bd-4859-b021-b32c2e831a47 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:48.662883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508da6d-beed-48d3-b440-6fccfd24c04f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:48.663520 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508da6d-beed-48d3-b440-6fccfd24c04f service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:48.663802 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508da6d-beed-48d3-b440-6fccfd24c04f service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:48.664074 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508da6d-beed-48d3-b440-6fccfd24c04f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:48.673906 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508da6d-beed-48d3-b440-6fccfd24c04f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:48.675553 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508da6d-beed-48d3-b440-6fccfd24c04f service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:48.685134 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 721/1425] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:57:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:49.611155 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0ffc4-7d8d-47c8-960f-f50cc7027c74 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:49.611723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0ffc4-7d8d-47c8-960f-f50cc7027c74 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:49.611723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0ffc4-7d8d-47c8-960f-f50cc7027c74 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:49.613102 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 17:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:49.618710 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227b72e-2c71-4b3b-8f89-f44e3c8377b8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:49.619132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227b72e-2c71-4b3b-8f89-f44e3c8377b8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:49.619270 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227b72e-2c71-4b3b-8f89-f44e3c8377b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:49.638767 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c227b72e-2c71-4b3b-8f89-f44e3c8377b8 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:49.639968 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c227b72e-2c71-4b3b-8f89-f44e3c8377b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:49.647697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 17:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:49.795884 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0ffc4-7d8d-47c8-960f-f50cc7027c74 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:49.811034 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0ffc4-7d8d-47c8-960f-f50cc7027c74 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:49.816456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c772c22-62a3-4a0f-a0cf-96debdaab11a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:49.817107 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c772c22-62a3-4a0f-a0cf-96debdaab11a service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:49.817395 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c772c22-62a3-4a0f-a0cf-96debdaab11a service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:49.817678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c772c22-62a3-4a0f-a0cf-96debdaab11a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:49.826065 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c772c22-62a3-4a0f-a0cf-96debdaab11a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:49.827715 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c772c22-62a3-4a0f-a0cf-96debdaab11a service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:49.837802 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 706/1428] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:57:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:50.728504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0844115-3219-4fee-bbc6-f420c7a077a8 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:50.729023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0844115-3219-4fee-bbc6-f420c7a077a8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:50.729023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0844115-3219-4fee-bbc6-f420c7a077a8 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:50.730497 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 17:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:50.736517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f2f3b-da6c-4c20-8d2e-9ec5ef8b7d92 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:50.736941 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f2f3b-da6c-4c20-8d2e-9ec5ef8b7d92 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:50.737090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f2f3b-da6c-4c20-8d2e-9ec5ef8b7d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:50.755663 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-7e9f2f3b-da6c-4c20-8d2e-9ec5ef8b7d92 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:50.756906 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-7e9f2f3b-da6c-4c20-8d2e-9ec5ef8b7d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:50.765426 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 17:57:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:50.916733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0844115-3219-4fee-bbc6-f420c7a077a8 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:50.932272 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0844115-3219-4fee-bbc6-f420c7a077a8 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:50.937743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c11b5b1-3def-41b1-836c-802398b7d923 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:50.938447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11b5b1-3def-41b1-836c-802398b7d923 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:50.938736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11b5b1-3def-41b1-836c-802398b7d923 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:50.939012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11b5b1-3def-41b1-836c-802398b7d923 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:50.948970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c11b5b1-3def-41b1-836c-802398b7d923 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:50.950607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c11b5b1-3def-41b1-836c-802398b7d923 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:50.960075 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 724/1431] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:57:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:51.820382 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f046a0-5cd2-4d61-9a6b-cd7aa2d00416 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:51.821717 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f046a0-5cd2-4d61-9a6b-cd7aa2d00416 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:51.821717 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f046a0-5cd2-4d61-9a6b-cd7aa2d00416 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:51.822963 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 17:57:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:51.828025 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28c684-3534-434d-aaad-9b183b106ec3 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:51.828483 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28c684-3534-434d-aaad-9b183b106ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:51.828606 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28c684-3534-434d-aaad-9b183b106ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:51.847285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-9b28c684-3534-434d-aaad-9b183b106ec3 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:51.848473 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9b28c684-3534-434d-aaad-9b183b106ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:51.856406 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 17:57:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:52.010115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f046a0-5cd2-4d61-9a6b-cd7aa2d00416 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:52.025488 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f046a0-5cd2-4d61-9a6b-cd7aa2d00416 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:52.030869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d2016-a5a6-4bc0-b1c4-2bf5f244d225 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:52.031512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d2016-a5a6-4bc0-b1c4-2bf5f244d225 service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:52.031797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d2016-a5a6-4bc0-b1c4-2bf5f244d225 service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:52.032132 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d2016-a5a6-4bc0-b1c4-2bf5f244d225 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:52.040931 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d2016-a5a6-4bc0-b1c4-2bf5f244d225 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:52.042550 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d2016-a5a6-4bc0-b1c4-2bf5f244d225 service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:52.052442 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 709/1434] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:57:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:52.907886 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6900b1e-f470-4713-87e5-7eeb8757fd59 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:52.908701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6900b1e-f470-4713-87e5-7eeb8757fd59 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:52.908701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6900b1e-f470-4713-87e5-7eeb8757fd59 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:52.909979 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 17:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:52.915418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91a524-32bf-4534-ae9e-569461f2bc6a None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:52.915839 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91a524-32bf-4534-ae9e-569461f2bc6a None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:52.916011 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91a524-32bf-4534-ae9e-569461f2bc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:52.934600 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-8e91a524-32bf-4534-ae9e-569461f2bc6a None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:52.935794 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-8e91a524-32bf-4534-ae9e-569461f2bc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:52.943496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 17:57:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:53.094086 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6900b1e-f470-4713-87e5-7eeb8757fd59 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:53.109178 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6900b1e-f470-4713-87e5-7eeb8757fd59 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:53.114535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f738e1f-d475-4a6a-95e5-b556075e05ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:53.115174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f738e1f-d475-4a6a-95e5-b556075e05ea service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:53.115456 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f738e1f-d475-4a6a-95e5-b556075e05ea service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:53.115731 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f738e1f-d475-4a6a-95e5-b556075e05ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:53.124418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f738e1f-d475-4a6a-95e5-b556075e05ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:53.125992 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f738e1f-d475-4a6a-95e5-b556075e05ea service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:53.135861 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 727/1437] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:57:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:53.986622 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a87cdf-9744-4b88-8ecc-0435061e8ee8 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:53.987162 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a87cdf-9744-4b88-8ecc-0435061e8ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:53.987162 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a87cdf-9744-4b88-8ecc-0435061e8ee8 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:53.988500 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 17:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:53.993978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0e0e3-6421-4e2d-9d4b-ff0e0f1ee465 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:53.994435 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0e0e3-6421-4e2d-9d4b-ff0e0f1ee465 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:53.994552 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0e0e3-6421-4e2d-9d4b-ff0e0f1ee465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:54.013608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c8e0e0e3-6421-4e2d-9d4b-ff0e0f1ee465 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:54.014851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c8e0e0e3-6421-4e2d-9d4b-ff0e0f1ee465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:54.022675 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 17:57:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:54.172375 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a87cdf-9744-4b88-8ecc-0435061e8ee8 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:54.188160 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a87cdf-9744-4b88-8ecc-0435061e8ee8 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:54.194383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ff886-e221-4154-8da0-faed0d82e4f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:54.195174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ff886-e221-4154-8da0-faed0d82e4f3 service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:54.195458 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ff886-e221-4154-8da0-faed0d82e4f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:54.195737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ff886-e221-4154-8da0-faed0d82e4f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:54.211984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ff886-e221-4154-8da0-faed0d82e4f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:54.213663 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ff886-e221-4154-8da0-faed0d82e4f3 service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:54.223677 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 712/1440] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:57:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:56.326745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c5d00-aa4a-4dd0-b3b4-8473f1d76f56 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:56.327305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c5d00-aa4a-4dd0-b3b4-8473f1d76f56 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:56.327305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c5d00-aa4a-4dd0-b3b4-8473f1d76f56 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:56.328715 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:57:56.333999 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0ac52-158f-4b09-bd2a-a7f4e5491973 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:56.334551 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0ac52-158f-4b09-bd2a-a7f4e5491973 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:56.334646 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0ac52-158f-4b09-bd2a-a7f4e5491973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:56.352756 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-fac0ac52-158f-4b09-bd2a-a7f4e5491973 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:56.353953 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-fac0ac52-158f-4b09-bd2a-a7f4e5491973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:56.361620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:57:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:57:56.399983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c5d00-aa4a-4dd0-b3b4-8473f1d76f56 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:56.410946 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c5d00-aa4a-4dd0-b3b4-8473f1d76f56 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:56.414427 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4e652-5647-45c1-b6e5-744b70d583f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:56.415559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4e652-5647-45c1-b6e5-744b70d583f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:56.415837 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4e652-5647-45c1-b6e5-744b70d583f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:56.416114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4e652-5647-45c1-b6e5-744b70d583f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:56.424830 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4e652-5647-45c1-b6e5-744b70d583f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:56.426366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4e652-5647-45c1-b6e5-744b70d583f3 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:56.436140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 730/1443] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:57:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:57:58.792981 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789d278-0546-4244-a698-258bf83849fa None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:58.793563 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789d278-0546-4244-a698-258bf83849fa None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:58.793563 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789d278-0546-4244-a698-258bf83849fa None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:58.795292 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:57:58.800690 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424541ce-5a9d-4976-ac1f-4c2760c4bdc3 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:58.801156 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424541ce-5a9d-4976-ac1f-4c2760c4bdc3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:58.801261 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424541ce-5a9d-4976-ac1f-4c2760c4bdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:58.819138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-424541ce-5a9d-4976-ac1f-4c2760c4bdc3 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:57:58.820335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-424541ce-5a9d-4976-ac1f-4c2760c4bdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:57:58.828850 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:57:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:57:58.867180 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789d278-0546-4244-a698-258bf83849fa None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:57:58.878120 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789d278-0546-4244-a698-258bf83849fa None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:57:58.881833 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5226d-bee9-4766-a85d-0d72e96fbcc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:57:58.882513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5226d-bee9-4766-a85d-0d72e96fbcc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:57:58.882798 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5226d-bee9-4766-a85d-0d72e96fbcc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:57:58.883595 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5226d-bee9-4766-a85d-0d72e96fbcc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:57:58.892522 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5226d-bee9-4766-a85d-0d72e96fbcc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:57:58.893944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5226d-bee9-4766-a85d-0d72e96fbcc8 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:57:58.904530 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 715/1446] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:57:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:58:01.435201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe3450-77c6-4d7c-a5fc-e45883924c04 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:01.435822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe3450-77c6-4d7c-a5fc-e45883924c04 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:01.435822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe3450-77c6-4d7c-a5fc-e45883924c04 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:01.437105 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:58:01.442355 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e5e6e-a669-468e-aae6-a48ddb332b03 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:01.442811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e5e6e-a669-468e-aae6-a48ddb332b03 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:01.442920 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e5e6e-a669-468e-aae6-a48ddb332b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:01.460845 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-549e5e6e-a669-468e-aae6-a48ddb332b03 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:58:01.463046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-549e5e6e-a669-468e-aae6-a48ddb332b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:58:01.470761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:58:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:58:01.509788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe3450-77c6-4d7c-a5fc-e45883924c04 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:58:01.520518 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe3450-77c6-4d7c-a5fc-e45883924c04 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:58:01.524632 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444cd72b-f1e6-4984-b481-7bde5cc98e2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:58:01.525266 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444cd72b-f1e6-4984-b481-7bde5cc98e2c service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:01.525554 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444cd72b-f1e6-4984-b481-7bde5cc98e2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:01.525836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444cd72b-f1e6-4984-b481-7bde5cc98e2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:01.535069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444cd72b-f1e6-4984-b481-7bde5cc98e2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:58:01.536508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444cd72b-f1e6-4984-b481-7bde5cc98e2c service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:58:01.547469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 733/1449] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:58:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:58:03.941857 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd371e0a-8c3e-47d5-b9d9-4765bd10f20d None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:03.942419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd371e0a-8c3e-47d5-b9d9-4765bd10f20d None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:03.942419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd371e0a-8c3e-47d5-b9d9-4765bd10f20d None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:03.943757 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:58:03.949604 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce27e7a-bd74-4a59-9f36-f3379cd5c511 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:03.950027 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce27e7a-bd74-4a59-9f36-f3379cd5c511 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:03.950144 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce27e7a-bd74-4a59-9f36-f3379cd5c511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:03.968420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-fce27e7a-bd74-4a59-9f36-f3379cd5c511 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:58:03.969622 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fce27e7a-bd74-4a59-9f36-f3379cd5c511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:58:03.977585 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:58:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:58:04.016612 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd371e0a-8c3e-47d5-b9d9-4765bd10f20d None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:58:04.018397 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-fd371e0a-8c3e-47d5-b9d9-4765bd10f20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:58:04.031845 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd371e0a-8c3e-47d5-b9d9-4765bd10f20d None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:58:04.035690 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca552f-03f3-4401-8555-28edd1924922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:58:04.036747 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca552f-03f3-4401-8555-28edd1924922 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:04.036747 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca552f-03f3-4401-8555-28edd1924922 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:04.036962 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca552f-03f3-4401-8555-28edd1924922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:04.046101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca552f-03f3-4401-8555-28edd1924922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:58:04.047588 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca552f-03f3-4401-8555-28edd1924922 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:58:04.057316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 718/1452] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:58:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:58:06.420920 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34aae0-dd76-4eee-8946-e9d335deb9e2 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:06.421476 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34aae0-dd76-4eee-8946-e9d335deb9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:06.421476 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34aae0-dd76-4eee-8946-e9d335deb9e2 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:06.422864 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:58:06.428497 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7133e9f-513b-4637-b345-43d848321b00 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:06.428935 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7133e9f-513b-4637-b345-43d848321b00 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:06.429066 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7133e9f-513b-4637-b345-43d848321b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:06.447686 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-c7133e9f-513b-4637-b345-43d848321b00 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:58:06.448881 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-c7133e9f-513b-4637-b345-43d848321b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:58:06.456674 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:58:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:58:06.495091 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34aae0-dd76-4eee-8946-e9d335deb9e2 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:58:06.505591 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34aae0-dd76-4eee-8946-e9d335deb9e2 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:58:06.509984 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c43108-27ec-4bfb-bd87-5ccd4f2c1ead service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:58:06.510682 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c43108-27ec-4bfb-bd87-5ccd4f2c1ead service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:06.510968 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c43108-27ec-4bfb-bd87-5ccd4f2c1ead service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:06.511247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c43108-27ec-4bfb-bd87-5ccd4f2c1ead service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:06.520487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c43108-27ec-4bfb-bd87-5ccd4f2c1ead service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:58:06.521921 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c43108-27ec-4bfb-bd87-5ccd4f2c1ead service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:58:06.531433 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 736/1455] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:58:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:58:08.809437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16591397-0add-442b-b49e-bfcab96c9a34 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:08.810037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16591397-0add-442b-b49e-bfcab96c9a34 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:08.810037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16591397-0add-442b-b49e-bfcab96c9a34 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:08.811389 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:58:08.816770 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722243c-4f6c-4839-ac45-3deec3774cfb None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:08.817223 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722243c-4f6c-4839-ac45-3deec3774cfb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:08.817370 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722243c-4f6c-4839-ac45-3deec3774cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:08.836439 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-9722243c-4f6c-4839-ac45-3deec3774cfb None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:58:08.837650 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9722243c-4f6c-4839-ac45-3deec3774cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:58:08.845418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:58:08.882760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16591397-0add-442b-b49e-bfcab96c9a34 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:58:08.894359 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16591397-0add-442b-b49e-bfcab96c9a34 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:58:08.897957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5f98e-d4d1-439e-b5a0-a5d5c97ebbbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:58:08.898757 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5f98e-d4d1-439e-b5a0-a5d5c97ebbbf service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:08.899200 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5f98e-d4d1-439e-b5a0-a5d5c97ebbbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:08.899200 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5f98e-d4d1-439e-b5a0-a5d5c97ebbbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:08.908396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb5f98e-d4d1-439e-b5a0-a5d5c97ebbbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:58:08.909830 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb5f98e-d4d1-439e-b5a0-a5d5c97ebbbf service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:58:08.919454 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 721/1458] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:58:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:58:11.213698 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553df27c-5229-4e27-aba5-0706286baa3a None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:11.214276 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553df27c-5229-4e27-aba5-0706286baa3a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:11.214276 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553df27c-5229-4e27-aba5-0706286baa3a None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:11.215687 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:58:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:58:11.221219 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb8393-f0df-4100-84a3-57c178cabe2f None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:11.221661 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb8393-f0df-4100-84a3-57c178cabe2f None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:11.221788 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb8393-f0df-4100-84a3-57c178cabe2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:11.228581 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=70 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:58:11.241331 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-9beb8393-f0df-4100-84a3-57c178cabe2f None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:58:11.242578 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-9beb8393-f0df-4100-84a3-57c178cabe2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:58:11.250400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:58:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:58:11.288969 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553df27c-5229-4e27-aba5-0706286baa3a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:58:11.294961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=73 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:58:11.301673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553df27c-5229-4e27-aba5-0706286baa3a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:58:11.306013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b7eea-334c-460f-8aec-69dcd3c7b852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:58:11.306622 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b7eea-334c-460f-8aec-69dcd3c7b852 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:11.306903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b7eea-334c-460f-8aec-69dcd3c7b852 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:11.307205 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b7eea-334c-460f-8aec-69dcd3c7b852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:11.316264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b7eea-334c-460f-8aec-69dcd3c7b852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:58:11.317695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b7eea-334c-460f-8aec-69dcd3c7b852 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:58:11.327910 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 739/1461] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:58:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:58:13.788918 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456b4bf-c19b-448e-9f6d-796d8d626095 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:13.789718 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456b4bf-c19b-448e-9f6d-796d8d626095 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:13.789718 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456b4bf-c19b-448e-9f6d-796d8d626095 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:13.791058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:58:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:58:13.796919 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082b348-3acc-4ce1-858e-eaac043206ab None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:13.797452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082b348-3acc-4ce1-858e-eaac043206ab None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:13.797579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082b348-3acc-4ce1-858e-eaac043206ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:13.816071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6082b348-3acc-4ce1-858e-eaac043206ab None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:58:13.817280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6082b348-3acc-4ce1-858e-eaac043206ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:58:13.826167 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:58:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:58:13.869373 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456b4bf-c19b-448e-9f6d-796d8d626095 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:58:13.881219 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456b4bf-c19b-448e-9f6d-796d8d626095 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:58:13.885449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c244c-e93a-4689-bbc5-27917687ec42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:58:13.886082 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c244c-e93a-4689-bbc5-27917687ec42 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:13.886406 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c244c-e93a-4689-bbc5-27917687ec42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:13.886819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c244c-e93a-4689-bbc5-27917687ec42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:13.896738 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c244c-e93a-4689-bbc5-27917687ec42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:58:13.898563 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c244c-e93a-4689-bbc5-27917687ec42 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:58:13.910595 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 724/1464] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:58:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:58:16.414977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9babbdd-5de8-4f71-b3ef-2d8218eec098 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:16.415489 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9babbdd-5de8-4f71-b3ef-2d8218eec098 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:16.415489 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9babbdd-5de8-4f71-b3ef-2d8218eec098 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:16.416826 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 17:58:16.422368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356a20f-54f8-45b6-8415-26ccbd2a7717 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:16.422797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356a20f-54f8-45b6-8415-26ccbd2a7717 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:16.423130 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356a20f-54f8-45b6-8415-26ccbd2a7717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:16.442826 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-b356a20f-54f8-45b6-8415-26ccbd2a7717 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:58:16.444339 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-b356a20f-54f8-45b6-8415-26ccbd2a7717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:58:16.453544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:58:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:58:16.491595 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9babbdd-5de8-4f71-b3ef-2d8218eec098 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:58:16.504195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9babbdd-5de8-4f71-b3ef-2d8218eec098 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:58:16.507775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7b6ac-dd24-4967-8d58-926ead77fa19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:58:16.508435 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7b6ac-dd24-4967-8d58-926ead77fa19 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:16.508719 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7b6ac-dd24-4967-8d58-926ead77fa19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:16.508996 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7b6ac-dd24-4967-8d58-926ead77fa19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:16.518870 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7b6ac-dd24-4967-8d58-926ead77fa19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:58:16.520358 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7b6ac-dd24-4967-8d58-926ead77fa19 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:58:16.530508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 742/1467] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:58:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:58:18.826379 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670297c-3632-4eae-8035-1064bd3c92e7 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:18.827017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670297c-3632-4eae-8035-1064bd3c92e7 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:18.827017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670297c-3632-4eae-8035-1064bd3c92e7 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:18.828338 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 17:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:58:18.834150 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6dc716-dc2e-4ea8-8150-480caa7fa55c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:18.834804 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6dc716-dc2e-4ea8-8150-480caa7fa55c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:18.834804 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6dc716-dc2e-4ea8-8150-480caa7fa55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:18.853987 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ba6dc716-dc2e-4ea8-8150-480caa7fa55c None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:58:18.855214 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ba6dc716-dc2e-4ea8-8150-480caa7fa55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:58:18.862895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 17:58:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:58:18.903301 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670297c-3632-4eae-8035-1064bd3c92e7 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:58:18.916188 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670297c-3632-4eae-8035-1064bd3c92e7 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:58:18.919941 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe218f64-46a7-4aed-acff-aae4627e45da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:58:18.920574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe218f64-46a7-4aed-acff-aae4627e45da service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:18.920856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe218f64-46a7-4aed-acff-aae4627e45da service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:18.921128 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe218f64-46a7-4aed-acff-aae4627e45da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:18.931658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe218f64-46a7-4aed-acff-aae4627e45da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:58:18.933120 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe218f64-46a7-4aed-acff-aae4627e45da service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:58:18.942555 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 727/1470] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:58:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:58:28.848207 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=17 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:58:28.938397 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=17 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:58:53.404961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd08796-9915-4dfe-bf62-89f5127350f1 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:53.405504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd08796-9915-4dfe-bf62-89f5127350f1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:53.405504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd08796-9915-4dfe-bf62-89f5127350f1 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:53.406854 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 744/1471] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:58:53.412437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21e71e-c361-4aea-b0ab-c84977ea11f8 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:53.412879 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21e71e-c361-4aea-b0ab-c84977ea11f8 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:53.413029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21e71e-c361-4aea-b0ab-c84977ea11f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:53.431290 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-4b21e71e-c361-4aea-b0ab-c84977ea11f8 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:58:53.432495 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-4b21e71e-c361-4aea-b0ab-c84977ea11f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:58:53.440258 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 728/1472] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:58:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:58:53.488652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf2896-6b52-4d60-ad75-833f2195dc04 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:53.488951 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf2896-6b52-4d60-ad75-833f2195dc04 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:53.489231 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf2896-6b52-4d60-ad75-833f2195dc04 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:53.490643 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 745/1473] 10.222.0.76 () {54 vars in 962 bytes} [Thu May 26 17:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:58:53.496518 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebfd64-fe15-4793-8064-e447908ccde5 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:53.496822 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebfd64-fe15-4793-8064-e447908ccde5 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:53.497109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebfd64-fe15-4793-8064-e447908ccde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:53.515701 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-83ebfd64-fe15-4793-8064-e447908ccde5 None None] MFA Rules not processed for user `52f4c9fc949c4a2c951b755234848cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:58:53.516899 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-83ebfd64-fe15-4793-8064-e447908ccde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:58:53.523422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 729/1474] 10.222.0.76 () {58 vars in 1059 bytes} [Thu May 26 17:58:53 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:58:53.530647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf2896-6b52-4d60-ad75-833f2195dc04 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:58:53.541646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf2896-6b52-4d60-ad75-833f2195dc04 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:58:53.545216 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176d952-7c62-4432-bd4f-2ba1c67d9c6d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f4c9fc949c4a2c951b755234848cc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:58:53.545846 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176d952-7c62-4432-bd4f-2ba1c67d9c6d service glance] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:53.546132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176d952-7c62-4432-bd4f-2ba1c67d9c6d service glance] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:53.546443 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176d952-7c62-4432-bd4f-2ba1c67d9c6d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:53.555855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176d952-7c62-4432-bd4f-2ba1c67d9c6d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:58:53.557292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176d952-7c62-4432-bd4f-2ba1c67d9c6d service glance] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:58:53.566631 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 746/1475] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:58:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:58:55.887830 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e68180-eaf8-4c70-91cc-df979ad3567b None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:55.888389 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e68180-eaf8-4c70-91cc-df979ad3567b None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:55.888389 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e68180-eaf8-4c70-91cc-df979ad3567b None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:55.889739 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 730/1476] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:58:55.895056 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151a986-792c-410a-9f2e-19343b48ff4a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:55.895475 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151a986-792c-410a-9f2e-19343b48ff4a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:55.895612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151a986-792c-410a-9f2e-19343b48ff4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:55.915291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e151a986-792c-410a-9f2e-19343b48ff4a None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:58:55.916513 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e151a986-792c-410a-9f2e-19343b48ff4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:58:55.924379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 747/1477] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:58:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:58:55.947219 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e68180-eaf8-4c70-91cc-df979ad3567b None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:58:55.959387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e68180-eaf8-4c70-91cc-df979ad3567b None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:58:55.964474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5e478-0ed6-4521-acbc-28f923ff0794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:58:55.965060 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5e478-0ed6-4521-acbc-28f923ff0794 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:55.965352 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5e478-0ed6-4521-acbc-28f923ff0794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:55.965624 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5e478-0ed6-4521-acbc-28f923ff0794 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:55.966410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5e478-0ed6-4521-acbc-28f923ff0794 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:58:55.968233 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: WARNING py.warnings [None req-d6a5e478-0ed6-4521-acbc-28f923ff0794 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 17:58:55.968233 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: warnings.warn(msg) May 26 17:58:55.968233 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: May 26 17:58:55.969373 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5e478-0ed6-4521-acbc-28f923ff0794 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:58:55.988873 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 731/1478] 10.222.0.76 () {56 vars in 1108 bytes} [Thu May 26 17:58:55 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 17:58:58.230547 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9cb213-2185-4cc2-ab31-2c262b34d2c2 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:58.231079 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9cb213-2185-4cc2-ab31-2c262b34d2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:58.231079 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9cb213-2185-4cc2-ab31-2c262b34d2c2 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:58.232490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 748/1479] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:58:58.237881 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86472050-81c6-47e2-a202-4784de1c012d None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:58.238487 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86472050-81c6-47e2-a202-4784de1c012d None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:58.238487 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86472050-81c6-47e2-a202-4784de1c012d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:58.258450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-86472050-81c6-47e2-a202-4784de1c012d None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:58:58.259676 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-86472050-81c6-47e2-a202-4784de1c012d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:58:58.267736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 732/1480] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:58:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:58:58.311712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9cb213-2185-4cc2-ab31-2c262b34d2c2 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:58:58.325907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9cb213-2185-4cc2-ab31-2c262b34d2c2 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:58:58.331519 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fde07-a2ed-4f35-b780-90be1743bf08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:58:58.332159 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fde07-a2ed-4f35-b780-90be1743bf08 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:58:58.332446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fde07-a2ed-4f35-b780-90be1743bf08 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:58:58.332725 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fde07-a2ed-4f35-b780-90be1743bf08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:58:58.341075 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fde07-a2ed-4f35-b780-90be1743bf08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:58:58.342713 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fde07-a2ed-4f35-b780-90be1743bf08 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:58:58.352768 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 749/1481] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:58:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:00.766008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01efd8f-9ced-430e-a7ba-6564405b8ab2 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:00.766584 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01efd8f-9ced-430e-a7ba-6564405b8ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:00.766584 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01efd8f-9ced-430e-a7ba-6564405b8ab2 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:00.767955 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 733/1482] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:59:00.773886 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe849392-b5e6-45f8-abe4-d99e818d66a9 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:00.774325 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe849392-b5e6-45f8-abe4-d99e818d66a9 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:00.774475 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe849392-b5e6-45f8-abe4-d99e818d66a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:00.792432 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-fe849392-b5e6-45f8-abe4-d99e818d66a9 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:00.793654 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fe849392-b5e6-45f8-abe4-d99e818d66a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:00.801485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 750/1483] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:59:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 17:59:00.838659 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01efd8f-9ced-430e-a7ba-6564405b8ab2 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:00.849928 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01efd8f-9ced-430e-a7ba-6564405b8ab2 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:00.854004 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae111a-ee5a-4579-956c-5c233bc0665b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:00.854672 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae111a-ee5a-4579-956c-5c233bc0665b service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:00.854954 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae111a-ee5a-4579-956c-5c233bc0665b service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:00.855228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae111a-ee5a-4579-956c-5c233bc0665b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:00.863983 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ae111a-ee5a-4579-956c-5c233bc0665b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:00.865405 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ae111a-ee5a-4579-956c-5c233bc0665b service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:00.875625 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 734/1484] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:59:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:03.155173 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c13257-bdab-43d5-b55f-95277226310c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:03.155694 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c13257-bdab-43d5-b55f-95277226310c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:03.155801 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c13257-bdab-43d5-b55f-95277226310c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:03.157119 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 751/1485] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:59:03.164245 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22913640-3b81-4532-b62d-3fdf959bc5b5 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:03.164751 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22913640-3b81-4532-b62d-3fdf959bc5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:03.165267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22913640-3b81-4532-b62d-3fdf959bc5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:03.183698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-22913640-3b81-4532-b62d-3fdf959bc5b5 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:03.184893 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-22913640-3b81-4532-b62d-3fdf959bc5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:03.192584 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 735/1486] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:59:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:03.230822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c13257-bdab-43d5-b55f-95277226310c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:03.241588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c13257-bdab-43d5-b55f-95277226310c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:03.245339 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82556a6-8ceb-4e5a-a714-696bf9beb153 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:03.245978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82556a6-8ceb-4e5a-a714-696bf9beb153 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:03.246293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82556a6-8ceb-4e5a-a714-696bf9beb153 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:03.246576 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82556a6-8ceb-4e5a-a714-696bf9beb153 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:03.255612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82556a6-8ceb-4e5a-a714-696bf9beb153 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:03.257218 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82556a6-8ceb-4e5a-a714-696bf9beb153 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:03.266630 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 752/1487] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:59:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:05.579569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15626b27-1bd9-4f5c-b7d6-2a085fb17ec3 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:05.580146 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15626b27-1bd9-4f5c-b7d6-2a085fb17ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:05.580146 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15626b27-1bd9-4f5c-b7d6-2a085fb17ec3 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:05.581563 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 736/1488] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:59:05.587122 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48d8ba-998b-4968-a923-2c54c8474e3d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:05.587600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48d8ba-998b-4968-a923-2c54c8474e3d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:05.587704 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48d8ba-998b-4968-a923-2c54c8474e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:05.605596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ed48d8ba-998b-4968-a923-2c54c8474e3d None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:05.606859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ed48d8ba-998b-4968-a923-2c54c8474e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:05.614870 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 753/1489] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:59:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:05.657863 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15626b27-1bd9-4f5c-b7d6-2a085fb17ec3 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:05.670037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15626b27-1bd9-4f5c-b7d6-2a085fb17ec3 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:05.673674 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430a3f5-0554-4a6f-8129-49d3db40e37e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:05.674356 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430a3f5-0554-4a6f-8129-49d3db40e37e service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:05.674642 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430a3f5-0554-4a6f-8129-49d3db40e37e service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:05.674928 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430a3f5-0554-4a6f-8129-49d3db40e37e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:05.685021 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430a3f5-0554-4a6f-8129-49d3db40e37e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:05.686591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430a3f5-0554-4a6f-8129-49d3db40e37e service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:05.697162 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 737/1490] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:59:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:08.155818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc876e-4e0e-418c-9d98-a23f953bd6ae None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:08.156349 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc876e-4e0e-418c-9d98-a23f953bd6ae None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:08.156447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc876e-4e0e-418c-9d98-a23f953bd6ae None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:08.158030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 754/1491] 10.222.0.76 () {54 vars in 926 bytes} [Thu May 26 17:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 17:59:08.163388 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0cfc0-3137-4952-9fcc-c6e682d937fa None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:08.163820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0cfc0-3137-4952-9fcc-c6e682d937fa None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:08.163935 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0cfc0-3137-4952-9fcc-c6e682d937fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:08.182619 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-56b0cfc0-3137-4952-9fcc-c6e682d937fa None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 17:59:08.183828 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-56b0cfc0-3137-4952-9fcc-c6e682d937fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 17:59:08.191590 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 738/1492] 10.222.0.76 () {58 vars in 1023 bytes} [Thu May 26 17:59:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 17:59:08.230715 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc876e-4e0e-418c-9d98-a23f953bd6ae None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 17:59:08.242155 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc876e-4e0e-418c-9d98-a23f953bd6ae None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 17:59:08.245896 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75fc92-d72a-42cb-9897-fe72ee717be5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 17:59:08.246586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75fc92-d72a-42cb-9897-fe72ee717be5 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 17:59:08.246872 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75fc92-d72a-42cb-9897-fe72ee717be5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 17:59:08.247154 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75fc92-d72a-42cb-9897-fe72ee717be5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 17:59:08.257803 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75fc92-d72a-42cb-9897-fe72ee717be5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 17:59:08.259291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75fc92-d72a-42cb-9897-fe72ee717be5 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 17:59:08.269879 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 755/1493] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 17:59:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 17:59:18.177259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=33 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 17:59:18.265167 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=33 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:49.816931 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428b666-fb56-4e2c-8bf9-43e8d638ab1e None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:49.817528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428b666-fb56-4e2c-8bf9-43e8d638ab1e None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:49.817528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428b666-fb56-4e2c-8bf9-43e8d638ab1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:49.835853 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-b428b666-fb56-4e2c-8bf9-43e8d638ab1e None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:49.837053 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-b428b666-fb56-4e2c-8bf9-43e8d638ab1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:49.844905 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 739/1494] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:49.855047 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b42c6-afc8-4534-9952-b7116a1f8795 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:49.855588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b42c6-afc8-4534-9952-b7116a1f8795 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:49.855696 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b42c6-afc8-4534-9952-b7116a1f8795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:49.874181 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-243b42c6-afc8-4534-9952-b7116a1f8795 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:49.875416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-243b42c6-afc8-4534-9952-b7116a1f8795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:49.883564 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 756/1495] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:49.888302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428b666-fb56-4e2c-8bf9-43e8d638ab1e None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:49.899175 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428b666-fb56-4e2c-8bf9-43e8d638ab1e None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:49.903720 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b89ffb-7c75-45bb-99e2-d0bcfb588482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:49.904386 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b89ffb-7c75-45bb-99e2-d0bcfb588482 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:49.904675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b89ffb-7c75-45bb-99e2-d0bcfb588482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:49.904957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b89ffb-7c75-45bb-99e2-d0bcfb588482 admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:49.905688 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b89ffb-7c75-45bb-99e2-d0bcfb588482 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:00:49.906000 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b89ffb-7c75-45bb-99e2-d0bcfb588482 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:49.907340 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: WARNING py.warnings [None req-88b89ffb-7c75-45bb-99e2-d0bcfb588482 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 18:00:49.907340 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: warnings.warn(msg) May 26 18:00:49.907340 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: May 26 18:00:49.908247 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b89ffb-7c75-45bb-99e2-d0bcfb588482 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:49.921551 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 740/1496] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:00:49.929971 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b42c6-afc8-4534-9952-b7116a1f8795 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:49.940523 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b42c6-afc8-4534-9952-b7116a1f8795 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:49.945400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034744fa-b84a-456d-8610-642cc7a58c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:49.945975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034744fa-b84a-456d-8610-642cc7a58c20 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:49.946293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034744fa-b84a-456d-8610-642cc7a58c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:49.946579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034744fa-b84a-456d-8610-642cc7a58c20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:49.947277 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034744fa-b84a-456d-8610-642cc7a58c20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:49.949089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-034744fa-b84a-456d-8610-642cc7a58c20 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 18:00:49.949089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: warnings.warn(msg) May 26 18:00:49.949089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: May 26 18:00:49.950363 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034744fa-b84a-456d-8610-642cc7a58c20 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:49.994139 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 757/1497] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:00:49 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:49.995859 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b89ffb-7c75-45bb-99e2-d0bcfb588482 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:50.006147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b89ffb-7c75-45bb-99e2-d0bcfb588482 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:50.063228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc0829-5376-4b00-8971-6b5635edd2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:50.063785 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc0829-5376-4b00-8971-6b5635edd2d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:50.064102 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc0829-5376-4b00-8971-6b5635edd2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:50.064383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc0829-5376-4b00-8971-6b5635edd2d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:50.065085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc0829-5376-4b00-8971-6b5635edd2d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:50.066111 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: WARNING py.warnings [None req-83cc0829-5376-4b00-8971-6b5635edd2d8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 18:00:50.066111 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: warnings.warn(msg) May 26 18:00:50.066111 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: May 26 18:00:50.067895 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc0829-5376-4b00-8971-6b5635edd2d8 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:50.089591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 741/1498] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:00:49 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:00:50.096255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034744fa-b84a-456d-8610-642cc7a58c20 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:50.114742 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034744fa-b84a-456d-8610-642cc7a58c20 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:50.121133 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbc179-2e61-46db-84c1-7ed7e6ec43e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:50.121905 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbc179-2e61-46db-84c1-7ed7e6ec43e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:50.122254 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbc179-2e61-46db-84c1-7ed7e6ec43e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:50.122562 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbc179-2e61-46db-84c1-7ed7e6ec43e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:50.123397 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abbc179-2e61-46db-84c1-7ed7e6ec43e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:50.124452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-8abbc179-2e61-46db-84c1-7ed7e6ec43e7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 18:00:50.124452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: warnings.warn(msg) May 26 18:00:50.124452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: May 26 18:00:50.125456 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abbc179-2e61-46db-84c1-7ed7e6ec43e7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:50.140789 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 758/1499] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:50.145004 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc0829-5376-4b00-8971-6b5635edd2d8 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:50.155329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc0829-5376-4b00-8971-6b5635edd2d8 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:50.161275 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d36ab-efdd-47b8-b4a1-c5212a5652e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:50.162110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d36ab-efdd-47b8-b4a1-c5212a5652e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:50.162437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d36ab-efdd-47b8-b4a1-c5212a5652e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:50.162722 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d36ab-efdd-47b8-b4a1-c5212a5652e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:50.163457 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d36ab-efdd-47b8-b4a1-c5212a5652e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:50.164473 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: WARNING py.warnings [None req-352d36ab-efdd-47b8-b4a1-c5212a5652e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 18:00:50.164473 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: warnings.warn(msg) May 26 18:00:50.164473 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: May 26 18:00:50.165371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d36ab-efdd-47b8-b4a1-c5212a5652e5 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:50.181210 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 742/1500] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:00:50.185497 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbc179-2e61-46db-84c1-7ed7e6ec43e7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:50.195898 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbc179-2e61-46db-84c1-7ed7e6ec43e7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:50.200674 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fdcdd-72ee-4572-8fb0-4ca3939ce90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:50.201574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fdcdd-72ee-4572-8fb0-4ca3939ce90f admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:50.201858 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fdcdd-72ee-4572-8fb0-4ca3939ce90f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:50.202135 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fdcdd-72ee-4572-8fb0-4ca3939ce90f admin admin] PATH_INFO: `/v3/projects/8460c79276d7459db8a5ad3a0f56f06c/users/a0364b5b823a44648680eef5b64aa164/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:50.218933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9fdcdd-72ee-4572-8fb0-4ca3939ce90f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8460c79276d7459db8a5ad3a0f56f06c, user_id=a0364b5b823a44648680eef5b64aa164, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:50.219631 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-0c9fdcdd-72ee-4572-8fb0-4ca3939ce90f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 18:00:50.219631 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: warnings.warn(msg) May 26 18:00:50.219631 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: May 26 18:00:50.221614 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9fdcdd-72ee-4572-8fb0-4ca3939ce90f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:50.235860 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 759/1501] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:00:50 2022] PUT /identity/v3/projects/8460c79276d7459db8a5ad3a0f56f06c/users/a0364b5b823a44648680eef5b64aa164/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:50.248478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2e004-bc5a-4a95-b907-3f6c597e11e9 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:50.248770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2e004-bc5a-4a95-b907-3f6c597e11e9 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:50.249056 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2e004-bc5a-4a95-b907-3f6c597e11e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:50.277774 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-65c2e004-bc5a-4a95-b907-3f6c597e11e9 None None] MFA Rules not processed for user `a0364b5b823a44648680eef5b64aa164`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:50.279029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-65c2e004-bc5a-4a95-b907-3f6c597e11e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:50.348271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 743/1502] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:00:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:00:50.361235 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd8511-6299-4b68-a6e9-129a95252120 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:50.361526 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd8511-6299-4b68-a6e9-129a95252120 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:50.361807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd8511-6299-4b68-a6e9-129a95252120 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:50.363207 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 760/1503] 10.222.0.76 () {54 vars in 963 bytes} [Thu May 26 18:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:00:50.368888 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbc6a0-ef57-4f3d-baa6-3dd378244cbc None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:50.369181 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbc6a0-ef57-4f3d-baa6-3dd378244cbc None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:50.369524 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbc6a0-ef57-4f3d-baa6-3dd378244cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:50.387827 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-5bbbc6a0-ef57-4f3d-baa6-3dd378244cbc None None] MFA Rules not processed for user `76dbf8f3f1b44baca2d2aa90fe606a30`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:50.389031 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-5bbbc6a0-ef57-4f3d-baa6-3dd378244cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:50.439641 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:50.449216 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 744/1504] 10.222.0.76 () {58 vars in 1060 bytes} [Thu May 26 18:00:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:50.560223 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd8511-6299-4b68-a6e9-129a95252120 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:50.571388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd8511-6299-4b68-a6e9-129a95252120 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:50.574904 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07609a-a0ff-47dc-b0cb-a403d49c0da4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:50.575544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07609a-a0ff-47dc-b0cb-a403d49c0da4 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:50.575824 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07609a-a0ff-47dc-b0cb-a403d49c0da4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:50.576098 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07609a-a0ff-47dc-b0cb-a403d49c0da4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:50.585231 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07609a-a0ff-47dc-b0cb-a403d49c0da4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:50.586722 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07609a-a0ff-47dc-b0cb-a403d49c0da4 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:50.595748 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 761/1505] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:00:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:50.639895 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101348ee-1f24-4088-9709-d58111362186 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:50.640186 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101348ee-1f24-4088-9709-d58111362186 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:50.640468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101348ee-1f24-4088-9709-d58111362186 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:50.641758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 745/1506] 10.222.0.76 () {52 vars in 829 bytes} [Thu May 26 18:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:00:50.645971 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07609a-a0ff-47dc-b0cb-a403d49c0da4 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:50.710548 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07609a-a0ff-47dc-b0cb-a403d49c0da4 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:50.715110 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85aeb8b-4c10-4d48-b49f-72774cef8c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:50.715975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85aeb8b-4c10-4d48-b49f-72774cef8c73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:50.716256 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85aeb8b-4c10-4d48-b49f-72774cef8c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:50.716532 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85aeb8b-4c10-4d48-b49f-72774cef8c73 admin admin] PATH_INFO: `/v3/users/a0364b5b823a44648680eef5b64aa164` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:50.717702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85aeb8b-4c10-4d48-b49f-72774cef8c73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0364b5b823a44648680eef5b64aa164)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:50.718756 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-c85aeb8b-4c10-4d48-b49f-72774cef8c73 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 18:00:50.718756 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: warnings.warn(msg) May 26 18:00:50.718756 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: May 26 18:00:50.719936 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85aeb8b-4c10-4d48-b49f-72774cef8c73 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:50.787532 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c85aeb8b-4c10-4d48-b49f-72774cef8c73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0364b5b823a44648680eef5b64aa164'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:50.797072 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c85aeb8b-4c10-4d48-b49f-72774cef8c73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0364b5b823a44648680eef5b64aa164'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:50.817178 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c85aeb8b-4c10-4d48-b49f-72774cef8c73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0364b5b823a44648680eef5b64aa164'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:50.817965 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c85aeb8b-4c10-4d48-b49f-72774cef8c73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0364b5b823a44648680eef5b64aa164'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:50.818724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c85aeb8b-4c10-4d48-b49f-72774cef8c73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0364b5b823a44648680eef5b64aa164'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:50.827479 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:00:50.840696 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 762/1507] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:00:50 2022] DELETE /identity/v3/users/a0364b5b823a44648680eef5b64aa164 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:00:50.845183 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf7fc0-0b0a-45b3-ac4b-b064460d6301 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:50.845687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf7fc0-0b0a-45b3-ac4b-b064460d6301 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:50.845800 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf7fc0-0b0a-45b3-ac4b-b064460d6301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:50.864446 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-3cbf7fc0-0b0a-45b3-ac4b-b064460d6301 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:00:50.865596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-3cbf7fc0-0b0a-45b3-ac4b-b064460d6301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:50.925634 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 746/1508] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:00:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:00:50.935288 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85aeb8b-4c10-4d48-b49f-72774cef8c73 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:50.936742 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c85aeb8b-4c10-4d48-b49f-72774cef8c73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:50.995847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85aeb8b-4c10-4d48-b49f-72774cef8c73 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:50.999461 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813db576-b82f-4143-b958-c7f7eb46cfe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:51.000119 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813db576-b82f-4143-b958-c7f7eb46cfe1 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:51.000409 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813db576-b82f-4143-b958-c7f7eb46cfe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:51.000691 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813db576-b82f-4143-b958-c7f7eb46cfe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:51.010392 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813db576-b82f-4143-b958-c7f7eb46cfe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:51.011843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813db576-b82f-4143-b958-c7f7eb46cfe1 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:51.021814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 763/1509] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:00:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:00:51.380174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf7fc0-0b0a-45b3-ac4b-b064460d6301 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:00:51.381723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-3cbf7fc0-0b0a-45b3-ac4b-b064460d6301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:00:51.393650 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf7fc0-0b0a-45b3-ac4b-b064460d6301 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:00:51.398502 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff9288-d98c-4b52-ba1c-5d5118c93807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:00:51.399379 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff9288-d98c-4b52-ba1c-5d5118c93807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:00:51.399664 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff9288-d98c-4b52-ba1c-5d5118c93807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:00:51.399944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff9288-d98c-4b52-ba1c-5d5118c93807 admin admin] PATH_INFO: `/v3/projects/8460c79276d7459db8a5ad3a0f56f06c` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:00:51.401150 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff9288-d98c-4b52-ba1c-5d5118c93807 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8460c79276d7459db8a5ad3a0f56f06c)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:00:51.402174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: WARNING py.warnings [None req-f4ff9288-d98c-4b52-ba1c-5d5118c93807 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 18:00:51.402174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: warnings.warn(msg) May 26 18:00:51.402174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: May 26 18:00:51.403426 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff9288-d98c-4b52-ba1c-5d5118c93807 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:00:51.472443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f4ff9288-d98c-4b52-ba1c-5d5118c93807 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8460c79276d7459db8a5ad3a0f56f06c', 'request_id': 'req-f4ff9288-d98c-4b52-ba1c-5d5118c93807'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:51.482355 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f4ff9288-d98c-4b52-ba1c-5d5118c93807 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8460c79276d7459db8a5ad3a0f56f06c', 'request_id': 'req-f4ff9288-d98c-4b52-ba1c-5d5118c93807'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:51.489176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f4ff9288-d98c-4b52-ba1c-5d5118c93807 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8460c79276d7459db8a5ad3a0f56f06c', 'request_id': 'req-f4ff9288-d98c-4b52-ba1c-5d5118c93807'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:51.509098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f4ff9288-d98c-4b52-ba1c-5d5118c93807 admin admin] The token cache is being invalidate because project 8460c79276d7459db8a5ad3a0f56f06c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:00:51.509395 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f4ff9288-d98c-4b52-ba1c-5d5118c93807 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:51.510131 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f4ff9288-d98c-4b52-ba1c-5d5118c93807 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:00:51.511731 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 747/1510] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:00:51 2022] DELETE /identity/v3/projects/8460c79276d7459db8a5ad3a0f56f06c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:01:01.017069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=29 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:01:01.505935 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:35.054780 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00402d0-ba1b-46cb-b2d0-87727ca2352b None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:35.055447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00402d0-ba1b-46cb-b2d0-87727ca2352b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:35.055447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00402d0-ba1b-46cb-b2d0-87727ca2352b None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:35.056837 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 18:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:35.062333 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265df15-1573-4c0a-979c-19940699652f None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:35.062855 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265df15-1573-4c0a-979c-19940699652f None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:35.062961 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265df15-1573-4c0a-979c-19940699652f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:35.083819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-f265df15-1573-4c0a-979c-19940699652f None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:35.085101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f265df15-1573-4c0a-979c-19940699652f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:35.152989 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 18:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:02:35.331781 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00402d0-ba1b-46cb-b2d0-87727ca2352b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:35.333710 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f00402d0-ba1b-46cb-b2d0-87727ca2352b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:35.418745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00402d0-ba1b-46cb-b2d0-87727ca2352b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:35.425143 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db029f-de15-452e-ac1c-b7296b82e6a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:35.425829 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db029f-de15-452e-ac1c-b7296b82e6a2 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:35.426119 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db029f-de15-452e-ac1c-b7296b82e6a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:35.426432 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db029f-de15-452e-ac1c-b7296b82e6a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:35.435897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5db029f-de15-452e-ac1c-b7296b82e6a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:35.437554 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5db029f-de15-452e-ac1c-b7296b82e6a2 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:35.449919 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 765/1513] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:02:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:02:36.377358 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da33276-ef53-4300-80a4-c3b2fa603911 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:36.378400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da33276-ef53-4300-80a4-c3b2fa603911 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:36.378400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da33276-ef53-4300-80a4-c3b2fa603911 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:36.380167 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 749/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 18:02:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:36.386490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33099b2-53b8-419b-b855-6eb6cd8948a0 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:36.387041 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33099b2-53b8-419b-b855-6eb6cd8948a0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:36.387234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33099b2-53b8-419b-b855-6eb6cd8948a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:36.406668 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b33099b2-53b8-419b-b855-6eb6cd8948a0 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:36.407910 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b33099b2-53b8-419b-b855-6eb6cd8948a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:36.416452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 766/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 18:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:02:36.576936 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da33276-ef53-4300-80a4-c3b2fa603911 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:36.591893 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da33276-ef53-4300-80a4-c3b2fa603911 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:36.598769 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513695f6-cfbf-4902-87ff-c7cf2f4451f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:36.599407 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513695f6-cfbf-4902-87ff-c7cf2f4451f0 service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:36.599737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513695f6-cfbf-4902-87ff-c7cf2f4451f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:36.600060 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513695f6-cfbf-4902-87ff-c7cf2f4451f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:36.610767 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513695f6-cfbf-4902-87ff-c7cf2f4451f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:36.613287 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513695f6-cfbf-4902-87ff-c7cf2f4451f0 service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:36.624390 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 750/1516] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:02:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:02:36.907566 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f26afa-5405-4783-8eac-b2ebb36618e7 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:36.907933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f26afa-5405-4783-8eac-b2ebb36618e7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:36.908716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f26afa-5405-4783-8eac-b2ebb36618e7 None None] PATH_INFO: `/v3` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:36.910295 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 767/1517] 10.222.0.86 () {54 vars in 925 bytes} [Thu May 26 18:02:36 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:02:36.916758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5456f-dac3-4329-b8da-322b9a95ffe1 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:36.917048 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5456f-dac3-4329-b8da-322b9a95ffe1 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:36.917336 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5456f-dac3-4329-b8da-322b9a95ffe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:36.936916 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-d7d5456f-dac3-4329-b8da-322b9a95ffe1 None None] MFA Rules not processed for user `596f126382e3440fb3cf7c357eef889f`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:36.938115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-d7d5456f-dac3-4329-b8da-322b9a95ffe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:37.009456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 751/1518] 10.222.0.86 () {58 vars in 1015 bytes} [Thu May 26 18:02:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:02:46.400565 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=37 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:46.556341 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cebdd7-eabe-42dc-82a1-acbc709256eb None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:46.556629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cebdd7-eabe-42dc-82a1-acbc709256eb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:46.556909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cebdd7-eabe-42dc-82a1-acbc709256eb None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:46.558327 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 768/1519] 10.222.0.76 () {54 vars in 960 bytes} [Thu May 26 18:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:46.566816 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f042ff-2912-4ed4-ae77-fb2a7576135d None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:46.567417 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f042ff-2912-4ed4-ae77-fb2a7576135d None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:46.567782 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f042ff-2912-4ed4-ae77-fb2a7576135d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:46.598797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-24f042ff-2912-4ed4-ae77-fb2a7576135d None None] MFA Rules not processed for user `596f126382e3440fb3cf7c357eef889f`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:46.600696 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-24f042ff-2912-4ed4-ae77-fb2a7576135d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:46.617105 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 752/1520] 10.222.0.76 () {58 vars in 1057 bytes} [Thu May 26 18:02:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:02:46.630161 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cebdd7-eabe-42dc-82a1-acbc709256eb None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:46.645466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cebdd7-eabe-42dc-82a1-acbc709256eb None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:46.650949 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8bdba-aeab-4b7d-8f89-a9e553857326 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '596f126382e3440fb3cf7c357eef889f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:46.651581 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8bdba-aeab-4b7d-8f89-a9e553857326 service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:46.651913 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8bdba-aeab-4b7d-8f89-a9e553857326 service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:46.652230 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8bdba-aeab-4b7d-8f89-a9e553857326 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:46.662737 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8bdba-aeab-4b7d-8f89-a9e553857326 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:46.664437 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8bdba-aeab-4b7d-8f89-a9e553857326 service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:46.676597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 769/1521] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:02:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:02:48.045749 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6c4ec-da29-4c4d-ad9e-6333d72fcf7c None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:48.046352 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6c4ec-da29-4c4d-ad9e-6333d72fcf7c None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:48.046352 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6c4ec-da29-4c4d-ad9e-6333d72fcf7c None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:48.047760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 18:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:48.053664 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03935669-d67f-4f45-a8cf-c0546d69f3e3 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:48.054226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03935669-d67f-4f45-a8cf-c0546d69f3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:48.054226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03935669-d67f-4f45-a8cf-c0546d69f3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:48.073513 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-03935669-d67f-4f45-a8cf-c0546d69f3e3 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:48.074764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-03935669-d67f-4f45-a8cf-c0546d69f3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:48.083578 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 18:02:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:02:48.245768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c6c4ec-da29-4c4d-ad9e-6333d72fcf7c None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:48.261917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c6c4ec-da29-4c4d-ad9e-6333d72fcf7c None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:48.268712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc477f5f-6b7d-4541-9300-efef7d7829bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:48.269392 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc477f5f-6b7d-4541-9300-efef7d7829bc service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:48.269683 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc477f5f-6b7d-4541-9300-efef7d7829bc service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:48.269967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc477f5f-6b7d-4541-9300-efef7d7829bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:48.279873 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc477f5f-6b7d-4541-9300-efef7d7829bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:48.281842 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc477f5f-6b7d-4541-9300-efef7d7829bc service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:48.293814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 754/1524] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:02:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:02:49.193116 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37899c9-8ec7-4637-946e-c390fe543751 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:49.193695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37899c9-8ec7-4637-946e-c390fe543751 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:49.193968 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37899c9-8ec7-4637-946e-c390fe543751 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:49.196685 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 18:02:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:49.202657 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aefb4f-77bc-4110-a009-202cf61223e5 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:49.203261 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aefb4f-77bc-4110-a009-202cf61223e5 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:49.203462 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aefb4f-77bc-4110-a009-202cf61223e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:49.224334 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-76aefb4f-77bc-4110-a009-202cf61223e5 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:49.225602 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-76aefb4f-77bc-4110-a009-202cf61223e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:49.233782 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 18:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:02:49.389179 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37899c9-8ec7-4637-946e-c390fe543751 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:49.406340 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37899c9-8ec7-4637-946e-c390fe543751 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:49.412643 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33844962-8e12-4d24-b1e5-9b4e3f7f8dd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:49.413355 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33844962-8e12-4d24-b1e5-9b4e3f7f8dd0 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:49.413650 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33844962-8e12-4d24-b1e5-9b4e3f7f8dd0 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:49.413930 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33844962-8e12-4d24-b1e5-9b4e3f7f8dd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:49.423894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33844962-8e12-4d24-b1e5-9b4e3f7f8dd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:49.425663 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33844962-8e12-4d24-b1e5-9b4e3f7f8dd0 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:49.436703 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 772/1527] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:02:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:02:53.773321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372416e-7512-4c18-868f-80938d38acab None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:53.773898 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372416e-7512-4c18-868f-80938d38acab None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:53.773898 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372416e-7512-4c18-868f-80938d38acab None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:53.775311 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 18:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:53.781046 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2651ee7-047d-4d74-adec-15c3a3aa0948 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:53.781541 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2651ee7-047d-4d74-adec-15c3a3aa0948 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:53.781646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2651ee7-047d-4d74-adec-15c3a3aa0948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:53.802890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c2651ee7-047d-4d74-adec-15c3a3aa0948 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:53.804131 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c2651ee7-047d-4d74-adec-15c3a3aa0948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:53.812484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 18:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:02:53.854787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372416e-7512-4c18-868f-80938d38acab None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:53.856745 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-3372416e-7512-4c18-868f-80938d38acab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:53.922520 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372416e-7512-4c18-868f-80938d38acab None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:53.926893 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141f1cb-ec5c-445b-bae8-b2d3c3c55839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:53.927567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141f1cb-ec5c-445b-bae8-b2d3c3c55839 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:53.927864 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141f1cb-ec5c-445b-bae8-b2d3c3c55839 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:53.928155 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141f1cb-ec5c-445b-bae8-b2d3c3c55839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:53.937905 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3141f1cb-ec5c-445b-bae8-b2d3c3c55839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:53.939977 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3141f1cb-ec5c-445b-bae8-b2d3c3c55839 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:53.945276 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:02:53.951065 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 757/1530] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:02:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:02:56.239051 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e84d79-a8a1-4e10-b616-644169b15061 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:56.239592 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e84d79-a8a1-4e10-b616-644169b15061 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:56.239592 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e84d79-a8a1-4e10-b616-644169b15061 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:56.241019 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 18:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:02:56.247055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accca9e1-a299-4874-b37e-f5f090660000 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:56.247610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accca9e1-a299-4874-b37e-f5f090660000 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:56.247712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accca9e1-a299-4874-b37e-f5f090660000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:56.265884 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-accca9e1-a299-4874-b37e-f5f090660000 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:02:56.267258 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-accca9e1-a299-4874-b37e-f5f090660000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:56.275179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 18:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:02:56.312402 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e84d79-a8a1-4e10-b616-644169b15061 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:02:56.314123 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c1e84d79-a8a1-4e10-b616-644169b15061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:02:56.327351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e84d79-a8a1-4e10-b616-644169b15061 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:02:56.331019 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c196be-694f-4891-8cba-a35215d67760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:02:56.331742 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c196be-694f-4891-8cba-a35215d67760 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:02:56.331937 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c196be-694f-4891-8cba-a35215d67760 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:02:56.332213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c196be-694f-4891-8cba-a35215d67760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:02:56.341309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c196be-694f-4891-8cba-a35215d67760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:02:56.342811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c196be-694f-4891-8cba-a35215d67760 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:02:56.353910 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 775/1533] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:02:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:03:00.154005 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7a22f-d55e-4acd-9396-317994c74c69 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:00.154690 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7a22f-d55e-4acd-9396-317994c74c69 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:00.154690 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7a22f-d55e-4acd-9396-317994c74c69 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:00.156327 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 18:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:00.163631 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f06b5-1069-4737-9aa3-b4b2be598ca7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:00.164241 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f06b5-1069-4737-9aa3-b4b2be598ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:00.164241 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f06b5-1069-4737-9aa3-b4b2be598ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:00.311916 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b22f06b5-1069-4737-9aa3-b4b2be598ca7 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:00.313191 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b22f06b5-1069-4737-9aa3-b4b2be598ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:00.321535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 18:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:03:00.600089 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7a22f-d55e-4acd-9396-317994c74c69 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:00.602624 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-68f7a22f-d55e-4acd-9396-317994c74c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:00.660746 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7a22f-d55e-4acd-9396-317994c74c69 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:00.664332 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c14e968-c3d6-4551-87af-c0988ca142aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:00.664986 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14e968-c3d6-4551-87af-c0988ca142aa service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:00.665277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14e968-c3d6-4551-87af-c0988ca142aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:00.665564 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14e968-c3d6-4551-87af-c0988ca142aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:00.675703 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c14e968-c3d6-4551-87af-c0988ca142aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:00.677293 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c14e968-c3d6-4551-87af-c0988ca142aa service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:00.687570 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 760/1536] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:03:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:03:02.972853 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506f9f8-7f45-4992-a8e4-34479d9109e5 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:02.973397 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506f9f8-7f45-4992-a8e4-34479d9109e5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:02.973397 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506f9f8-7f45-4992-a8e4-34479d9109e5 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:02.974946 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 18:03:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:03:02.981485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89682f71-3522-409d-81d7-7d9bf09d2e60 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:02.982009 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89682f71-3522-409d-81d7-7d9bf09d2e60 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:02.982121 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89682f71-3522-409d-81d7-7d9bf09d2e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:03.002685 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-89682f71-3522-409d-81d7-7d9bf09d2e60 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:03:03.003905 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-89682f71-3522-409d-81d7-7d9bf09d2e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:03.011735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 18:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:03:03.291822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506f9f8-7f45-4992-a8e4-34479d9109e5 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:03:03.293601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7506f9f8-7f45-4992-a8e4-34479d9109e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:03:03.307889 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506f9f8-7f45-4992-a8e4-34479d9109e5 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:03:03.311570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819741e0-abcf-4ea3-b14b-11ab5bf96231 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:03:03.312237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819741e0-abcf-4ea3-b14b-11ab5bf96231 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:03:03.312524 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819741e0-abcf-4ea3-b14b-11ab5bf96231 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:03:03.312802 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819741e0-abcf-4ea3-b14b-11ab5bf96231 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:03:03.323789 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819741e0-abcf-4ea3-b14b-11ab5bf96231 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:03:03.325185 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819741e0-abcf-4ea3-b14b-11ab5bf96231 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:03:03.336318 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 778/1539] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:03:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:03:12.995678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=32 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:03:13.331887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=33 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:38.260178 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf3454-8d31-4cff-be77-fbf04fb8a9e1 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.261365 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf3454-8d31-4cff-be77-fbf04fb8a9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.261895 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf3454-8d31-4cff-be77-fbf04fb8a9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.278597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad368c0-dd0b-4f20-8b06-b58799ab0bfd None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.279861 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad368c0-dd0b-4f20-8b06-b58799ab0bfd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.280638 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad368c0-dd0b-4f20-8b06-b58799ab0bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.303624 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-aebf3454-8d31-4cff-be77-fbf04fb8a9e1 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:38.304898 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-aebf3454-8d31-4cff-be77-fbf04fb8a9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:38.316710 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-bad368c0-dd0b-4f20-8b06-b58799ab0bfd None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:38.318838 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bad368c0-dd0b-4f20-8b06-b58799ab0bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:38.333367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 779/1540] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:38.334094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 762/1541] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:38.334836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbadc40-3ad4-4b39-838d-c3ffeee84eee None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.335127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbadc40-3ad4-4b39-838d-c3ffeee84eee None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.335408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbadc40-3ad4-4b39-838d-c3ffeee84eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.336391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d60c4-f47f-4f65-844e-23bb249e38a3 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.337039 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d60c4-f47f-4f65-844e-23bb249e38a3 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.340874 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d60c4-f47f-4f65-844e-23bb249e38a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.371885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-9a9d60c4-f47f-4f65-844e-23bb249e38a3 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:38.373209 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-9a9d60c4-f47f-4f65-844e-23bb249e38a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:38.374571 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8fbadc40-3ad4-4b39-838d-c3ffeee84eee None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:38.375762 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8fbadc40-3ad4-4b39-838d-c3ffeee84eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:38.382956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 763/1542] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:38.384573 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c7b66-8996-4905-9bf2-22e8105c9b82 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.384872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c7b66-8996-4905-9bf2-22e8105c9b82 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.385154 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c7b66-8996-4905-9bf2-22e8105c9b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.385280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 780/1543] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:38.386650 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45d278-25ac-47e6-9f3a-8d854c9faac6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.387291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45d278-25ac-47e6-9f3a-8d854c9faac6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.387577 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45d278-25ac-47e6-9f3a-8d854c9faac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.409721 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-9c45d278-25ac-47e6-9f3a-8d854c9faac6 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:38.410995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9c45d278-25ac-47e6-9f3a-8d854c9faac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:38.411781 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-334c7b66-8996-4905-9bf2-22e8105c9b82 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:38.413220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-334c7b66-8996-4905-9bf2-22e8105c9b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:38.422271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 764/1544] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:38.424589 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1e806-92aa-40e1-adda-e5e6cdd8804e None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.424935 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 781/1545] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:38.425298 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1e806-92aa-40e1-adda-e5e6cdd8804e None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.425740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1e806-92aa-40e1-adda-e5e6cdd8804e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.430688 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc88bf-3d45-41a7-a60b-5e63182214fa None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.431047 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc88bf-3d45-41a7-a60b-5e63182214fa None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.431339 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc88bf-3d45-41a7-a60b-5e63182214fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.451468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-27e1e806-92aa-40e1-adda-e5e6cdd8804e None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:38.452576 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-27e1e806-92aa-40e1-adda-e5e6cdd8804e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:38.454549 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-33dc88bf-3d45-41a7-a60b-5e63182214fa None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:38.455724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-33dc88bf-3d45-41a7-a60b-5e63182214fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:38.464654 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 782/1546] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:38.466029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3344f-cea1-4767-b1f8-5fb9759333e8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.466356 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3344f-cea1-4767-b1f8-5fb9759333e8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.466636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3344f-cea1-4767-b1f8-5fb9759333e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.471278 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 765/1547] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:38.472333 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32f6d5-ea70-40b6-a003-b5b324e667e4 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.472627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32f6d5-ea70-40b6-a003-b5b324e667e4 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.472896 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32f6d5-ea70-40b6-a003-b5b324e667e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.493589 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-6c32f6d5-ea70-40b6-a003-b5b324e667e4 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:38.493875 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d2a3344f-cea1-4767-b1f8-5fb9759333e8 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:38.494862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-6c32f6d5-ea70-40b6-a003-b5b324e667e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:38.495120 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d2a3344f-cea1-4767-b1f8-5fb9759333e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:38.503037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 766/1548] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:38.503666 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 783/1549] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:38.504213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3344f-cea1-4767-b1f8-5fb9759333e8 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:38.510287 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61e7cf-a941-4055-858d-f240bb47666b None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.511642 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61e7cf-a941-4055-858d-f240bb47666b None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.512162 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61e7cf-a941-4055-858d-f240bb47666b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.515534 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3344f-cea1-4767-b1f8-5fb9759333e8 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:38.522795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be7833-a9a2-4905-97c3-05f326ee704e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:38.523933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be7833-a9a2-4905-97c3-05f326ee704e admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.524479 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be7833-a9a2-4905-97c3-05f326ee704e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.524964 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be7833-a9a2-4905-97c3-05f326ee704e admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.526326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be7833-a9a2-4905-97c3-05f326ee704e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:38.526865 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be7833-a9a2-4905-97c3-05f326ee704e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:38.528734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-e0be7833-a9a2-4905-97c3-05f326ee704e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 18:04:38.528734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: warnings.warn(msg) May 26 18:04:38.528734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: May 26 18:04:38.530727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be7833-a9a2-4905-97c3-05f326ee704e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:38.539311 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-4a61e7cf-a941-4055-858d-f240bb47666b None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:38.540560 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-4a61e7cf-a941-4055-858d-f240bb47666b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:38.549317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 767/1550] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:38.550731 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72370fb4-01a4-45f8-a637-fa97602dfe29 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.551015 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72370fb4-01a4-45f8-a637-fa97602dfe29 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.551292 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72370fb4-01a4-45f8-a637-fa97602dfe29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.559259 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 784/1551] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:38.560938 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925913f-b296-46e6-9b70-0b0295a41313 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.561487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925913f-b296-46e6-9b70-0b0295a41313 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.561905 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925913f-b296-46e6-9b70-0b0295a41313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.582614 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-72370fb4-01a4-45f8-a637-fa97602dfe29 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:38.583856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-72370fb4-01a4-45f8-a637-fa97602dfe29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:38.585775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a925913f-b296-46e6-9b70-0b0295a41313 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:38.587011 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a925913f-b296-46e6-9b70-0b0295a41313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:38.592761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 768/1552] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:38.593613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72370fb4-01a4-45f8-a637-fa97602dfe29 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:38.597091 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 785/1553] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:38.598328 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d3bee-7a16-44a3-afdc-baa53b64c613 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.598658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d3bee-7a16-44a3-afdc-baa53b64c613 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.598932 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d3bee-7a16-44a3-afdc-baa53b64c613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.604942 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72370fb4-01a4-45f8-a637-fa97602dfe29 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:38.610134 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183afcd-cc9e-47d8-8aa1-4baa344b4a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:38.610943 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183afcd-cc9e-47d8-8aa1-4baa344b4a35 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.611266 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183afcd-cc9e-47d8-8aa1-4baa344b4a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.611548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183afcd-cc9e-47d8-8aa1-4baa344b4a35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.612306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183afcd-cc9e-47d8-8aa1-4baa344b4a35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:38.612641 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183afcd-cc9e-47d8-8aa1-4baa344b4a35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:38.614353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183afcd-cc9e-47d8-8aa1-4baa344b4a35 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:38.620174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-733d3bee-7a16-44a3-afdc-baa53b64c613 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:38.621785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-733d3bee-7a16-44a3-afdc-baa53b64c613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:38.627661 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 769/1554] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:38.628586 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183afcd-cc9e-47d8-8aa1-4baa344b4a35 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:38.636310 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 786/1555] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:38.637465 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d3bee-7a16-44a3-afdc-baa53b64c613 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:38.645474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183afcd-cc9e-47d8-8aa1-4baa344b4a35 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:38.649424 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d3bee-7a16-44a3-afdc-baa53b64c613 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:38.650263 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944a85f-933a-442b-a80f-c47db232b2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:38.650972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944a85f-933a-442b-a80f-c47db232b2e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.651264 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944a85f-933a-442b-a80f-c47db232b2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.651548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944a85f-933a-442b-a80f-c47db232b2e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.652305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944a85f-933a-442b-a80f-c47db232b2e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:38.652598 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944a85f-933a-442b-a80f-c47db232b2e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:38.654244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944a85f-933a-442b-a80f-c47db232b2e8 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:38.654442 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb0f16-7c15-4092-8cbf-a0e9f0ff24a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:38.655121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb0f16-7c15-4092-8cbf-a0e9f0ff24a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.655661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb0f16-7c15-4092-8cbf-a0e9f0ff24a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.655932 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb0f16-7c15-4092-8cbf-a0e9f0ff24a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.656662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb0f16-7c15-4092-8cbf-a0e9f0ff24a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:38.656965 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb0f16-7c15-4092-8cbf-a0e9f0ff24a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:38.659416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb0f16-7c15-4092-8cbf-a0e9f0ff24a9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:38.669329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 770/1556] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:38.670471 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2f85d-5c96-4983-9546-afb54a819c12 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.670734 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2f85d-5c96-4983-9546-afb54a819c12 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.671020 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2f85d-5c96-4983-9546-afb54a819c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.681990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 787/1557] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:38.681990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb0f16-7c15-4092-8cbf-a0e9f0ff24a9 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:38.699306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb0f16-7c15-4092-8cbf-a0e9f0ff24a9 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:38.703895 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-d6b2f85d-5c96-4983-9546-afb54a819c12 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:38.705738 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-d6b2f85d-5c96-4983-9546-afb54a819c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:38.706969 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0dd61c-0f5a-436e-9f15-e9dffe4f69cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:38.707686 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0dd61c-0f5a-436e-9f15-e9dffe4f69cd admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.707975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0dd61c-0f5a-436e-9f15-e9dffe4f69cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.708279 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0dd61c-0f5a-436e-9f15-e9dffe4f69cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.709022 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0dd61c-0f5a-436e-9f15-e9dffe4f69cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:38.711228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0dd61c-0f5a-436e-9f15-e9dffe4f69cd admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:38.713728 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 771/1558] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:38.714583 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2f85d-5c96-4983-9546-afb54a819c12 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:38.724518 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2f85d-5c96-4983-9546-afb54a819c12 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:38.729602 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed89f1-6b7e-43e2-9ebd-81273429dd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:38.730589 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed89f1-6b7e-43e2-9ebd-81273429dd60 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.730998 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed89f1-6b7e-43e2-9ebd-81273429dd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.731285 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed89f1-6b7e-43e2-9ebd-81273429dd60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.732131 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed89f1-6b7e-43e2-9ebd-81273429dd60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:38.732439 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed89f1-6b7e-43e2-9ebd-81273429dd60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:38.734376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed89f1-6b7e-43e2-9ebd-81273429dd60 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:38.739849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 788/1559] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:04:38 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:38.741195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939db465-89ca-4f5c-8c9a-3b5fb383fbc4 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.741610 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939db465-89ca-4f5c-8c9a-3b5fb383fbc4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.741886 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939db465-89ca-4f5c-8c9a-3b5fb383fbc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.756290 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 772/1560] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:38.756290 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed89f1-6b7e-43e2-9ebd-81273429dd60 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:38.760328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed89f1-6b7e-43e2-9ebd-81273429dd60 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:38.761022 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-939db465-89ca-4f5c-8c9a-3b5fb383fbc4 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:38.762277 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-939db465-89ca-4f5c-8c9a-3b5fb383fbc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:38.834988 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0473f7b-6683-484e-b09c-221a8bf9c667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:38.835654 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0473f7b-6683-484e-b09c-221a8bf9c667 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.835770 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 789/1561] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:38.835945 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0473f7b-6683-484e-b09c-221a8bf9c667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.836217 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0473f7b-6683-484e-b09c-221a8bf9c667 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.837362 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939db465-89ca-4f5c-8c9a-3b5fb383fbc4 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:38.837487 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0473f7b-6683-484e-b09c-221a8bf9c667 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:38.838027 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: WARNING py.warnings [None req-b0473f7b-6683-484e-b09c-221a8bf9c667 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 18:04:38.838027 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: warnings.warn(msg) May 26 18:04:38.838027 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: May 26 18:04:38.839708 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0473f7b-6683-484e-b09c-221a8bf9c667 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:38.847506 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939db465-89ca-4f5c-8c9a-3b5fb383fbc4 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:38.852108 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e91ed-61ae-4c35-9192-d5719c190c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:38.852771 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e91ed-61ae-4c35-9192-d5719c190c4a admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.853056 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e91ed-61ae-4c35-9192-d5719c190c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.853329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e91ed-61ae-4c35-9192-d5719c190c4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.854054 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e91ed-61ae-4c35-9192-d5719c190c4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:38.854520 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e91ed-61ae-4c35-9192-d5719c190c4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:38.856139 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e91ed-61ae-4c35-9192-d5719c190c4a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:38.869070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 790/1562] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:38.869643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 773/1563] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:04:38 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:38.870426 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0473f7b-6683-484e-b09c-221a8bf9c667 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:38.873397 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e91ed-61ae-4c35-9192-d5719c190c4a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:38.883987 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e91ed-61ae-4c35-9192-d5719c190c4a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:38.935141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:38.937262 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:38.948150 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0473f7b-6683-484e-b09c-221a8bf9c667 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:38.948485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b77e2e-89a9-460e-a418-6083481059e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:38.949227 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b77e2e-89a9-460e-a418-6083481059e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.949601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b77e2e-89a9-460e-a418-6083481059e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.949881 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b77e2e-89a9-460e-a418-6083481059e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.950858 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b77e2e-89a9-460e-a418-6083481059e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:38.953173 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b77e2e-89a9-460e-a418-6083481059e5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:38.953637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142d076-c62e-4c5a-a493-7d6031454f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:38.955154 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142d076-c62e-4c5a-a493-7d6031454f9e admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:38.955465 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142d076-c62e-4c5a-a493-7d6031454f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:38.955735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142d076-c62e-4c5a-a493-7d6031454f9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:38.956528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5142d076-c62e-4c5a-a493-7d6031454f9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:38.958502 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5142d076-c62e-4c5a-a493-7d6031454f9e admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:38.988121 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 774/1564] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:04:38 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:38.988977 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142d076-c62e-4c5a-a493-7d6031454f9e admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:38.993980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 791/1565] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:04:38 2022] POST /identity/v3/projects => generated 375 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:38.995779 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b77e2e-89a9-460e-a418-6083481059e5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.059223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142d076-c62e-4c5a-a493-7d6031454f9e admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.072830 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b77e2e-89a9-460e-a418-6083481059e5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.077987 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbe28d-46a4-439a-8eb6-2b0b7f5da40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.078737 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbe28d-46a4-439a-8eb6-2b0b7f5da40b admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.079070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbe28d-46a4-439a-8eb6-2b0b7f5da40b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.079925 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbe28d-46a4-439a-8eb6-2b0b7f5da40b admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.080636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbe28d-46a4-439a-8eb6-2b0b7f5da40b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.081689 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-c3cbe28d-46a4-439a-8eb6-2b0b7f5da40b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 18:04:39.081689 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: warnings.warn(msg) May 26 18:04:39.081689 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: May 26 18:04:39.082949 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbe28d-46a4-439a-8eb6-2b0b7f5da40b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.106715 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 792/1566] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:04:38 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:39.111190 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbe28d-46a4-439a-8eb6-2b0b7f5da40b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.121363 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbe28d-46a4-439a-8eb6-2b0b7f5da40b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.123459 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb168d0-f5b4-49d3-8bef-fa99298d48b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.124148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb168d0-f5b4-49d3-8bef-fa99298d48b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.124435 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb168d0-f5b4-49d3-8bef-fa99298d48b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.124715 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb168d0-f5b4-49d3-8bef-fa99298d48b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.125452 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb168d0-f5b4-49d3-8bef-fa99298d48b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:39.125762 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb168d0-f5b4-49d3-8bef-fa99298d48b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.126501 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018c18a-e10a-47bc-b8be-bb0befa37f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.127175 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018c18a-e10a-47bc-b8be-bb0befa37f69 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.127579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018c18a-e10a-47bc-b8be-bb0befa37f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.127864 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018c18a-e10a-47bc-b8be-bb0befa37f69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.127969 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb168d0-f5b4-49d3-8bef-fa99298d48b1 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.130270 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018c18a-e10a-47bc-b8be-bb0befa37f69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.130494 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018c18a-e10a-47bc-b8be-bb0befa37f69 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.141259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 775/1567] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:39.142559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb168d0-f5b4-49d3-8bef-fa99298d48b1 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.154430 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb168d0-f5b4-49d3-8bef-fa99298d48b1 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.165128 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 793/1568] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:04:39 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:39.167506 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d850a9e-d2ee-4953-bdc7-db8468824429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.168202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d850a9e-d2ee-4953-bdc7-db8468824429 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.168489 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d850a9e-d2ee-4953-bdc7-db8468824429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.168764 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d850a9e-d2ee-4953-bdc7-db8468824429 admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.169293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018c18a-e10a-47bc-b8be-bb0befa37f69 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.169495 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d850a9e-d2ee-4953-bdc7-db8468824429 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:39.169801 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d850a9e-d2ee-4953-bdc7-db8468824429 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.171451 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d850a9e-d2ee-4953-bdc7-db8468824429 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.188203 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 776/1569] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:39.189773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d850a9e-d2ee-4953-bdc7-db8468824429 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.203366 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d850a9e-d2ee-4953-bdc7-db8468824429 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.270509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b986c5-c9a2-4954-945b-9e1ac00a6593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.271265 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b986c5-c9a2-4954-945b-9e1ac00a6593 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.271472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b986c5-c9a2-4954-945b-9e1ac00a6593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.271754 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b986c5-c9a2-4954-945b-9e1ac00a6593 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.272355 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018c18a-e10a-47bc-b8be-bb0befa37f69 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.272484 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b986c5-c9a2-4954-945b-9e1ac00a6593 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.274416 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b986c5-c9a2-4954-945b-9e1ac00a6593 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.277228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795dadf0-27f1-46bb-acde-59f4832e7881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.277923 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795dadf0-27f1-46bb-acde-59f4832e7881 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.278223 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795dadf0-27f1-46bb-acde-59f4832e7881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.278515 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795dadf0-27f1-46bb-acde-59f4832e7881 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.279218 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795dadf0-27f1-46bb-acde-59f4832e7881 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.281471 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795dadf0-27f1-46bb-acde-59f4832e7881 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.290773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:39.307702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 794/1570] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:04:39 2022] POST /identity/v3/users => generated 292 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:39.309317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795dadf0-27f1-46bb-acde-59f4832e7881 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.310983 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 777/1571] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:04:39 2022] POST /identity/v3/projects => generated 369 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:39.312277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b986c5-c9a2-4954-945b-9e1ac00a6593 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.326821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795dadf0-27f1-46bb-acde-59f4832e7881 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.370159 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:39.384903 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b986c5-c9a2-4954-945b-9e1ac00a6593 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.389947 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37f1d2-accd-4fb6-a888-f6259fafb1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.390689 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37f1d2-accd-4fb6-a888-f6259fafb1bd admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.390969 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37f1d2-accd-4fb6-a888-f6259fafb1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.391404 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37f1d2-accd-4fb6-a888-f6259fafb1bd admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.392898 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37f1d2-accd-4fb6-a888-f6259fafb1bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.394879 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37f1d2-accd-4fb6-a888-f6259fafb1bd admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.415534 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb18ae4-6005-4b20-ae25-09969f20fb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.416227 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb18ae4-6005-4b20-ae25-09969f20fb05 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.416522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb18ae4-6005-4b20-ae25-09969f20fb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.416794 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb18ae4-6005-4b20-ae25-09969f20fb05 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.417483 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb18ae4-6005-4b20-ae25-09969f20fb05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.419440 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb18ae4-6005-4b20-ae25-09969f20fb05 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.419774 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 778/1572] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:04:39 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:39.420803 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37f1d2-accd-4fb6-a888-f6259fafb1bd admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.433921 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37f1d2-accd-4fb6-a888-f6259fafb1bd admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.440332 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9103d-efdb-43ea-813f-64fa1213ce7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.441106 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9103d-efdb-43ea-813f-64fa1213ce7c admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.441437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9103d-efdb-43ea-813f-64fa1213ce7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.441754 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9103d-efdb-43ea-813f-64fa1213ce7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.442566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9103d-efdb-43ea-813f-64fa1213ce7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.442862 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 795/1573] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:04:39 2022] POST /identity/v3/users => generated 292 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:39.443837 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb18ae4-6005-4b20-ae25-09969f20fb05 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.444215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9103d-efdb-43ea-813f-64fa1213ce7c admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.456183 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb18ae4-6005-4b20-ae25-09969f20fb05 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.461111 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 779/1574] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:39.461111 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9103d-efdb-43ea-813f-64fa1213ce7c admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.464795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9573f2c-190d-4cac-b4f6-c4a96a4c4c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.465901 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9573f2c-190d-4cac-b4f6-c4a96a4c4c0b admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.466299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9573f2c-190d-4cac-b4f6-c4a96a4c4c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.466677 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9573f2c-190d-4cac-b4f6-c4a96a4c4c0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.467550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9573f2c-190d-4cac-b4f6-c4a96a4c4c0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.469597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9573f2c-190d-4cac-b4f6-c4a96a4c4c0b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.470232 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9103d-efdb-43ea-813f-64fa1213ce7c admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.474618 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf25f9e-6bc0-4d14-b3dc-8a5588c70958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.475281 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf25f9e-6bc0-4d14-b3dc-8a5588c70958 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.475568 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf25f9e-6bc0-4d14-b3dc-8a5588c70958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.475846 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf25f9e-6bc0-4d14-b3dc-8a5588c70958 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.476533 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf25f9e-6bc0-4d14-b3dc-8a5588c70958 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.478470 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf25f9e-6bc0-4d14-b3dc-8a5588c70958 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.499101 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 796/1575] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:04:39 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:39.499806 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9573f2c-190d-4cac-b4f6-c4a96a4c4c0b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.508333 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 780/1576] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:04:39 2022] POST /identity/v3/users => generated 289 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:39.508333 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf25f9e-6bc0-4d14-b3dc-8a5588c70958 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.517943 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf25f9e-6bc0-4d14-b3dc-8a5588c70958 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.573608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9573f2c-190d-4cac-b4f6-c4a96a4c4c0b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.581188 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94bc126-881f-4de9-a356-aa5ef6ca4c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.581869 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94bc126-881f-4de9-a356-aa5ef6ca4c24 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.582151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94bc126-881f-4de9-a356-aa5ef6ca4c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.582451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94bc126-881f-4de9-a356-aa5ef6ca4c24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.583149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94bc126-881f-4de9-a356-aa5ef6ca4c24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.585209 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94bc126-881f-4de9-a356-aa5ef6ca4c24 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.586161 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be91ca-9744-4e53-af17-4cedda32d7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.586871 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be91ca-9744-4e53-af17-4cedda32d7fb admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.587158 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be91ca-9744-4e53-af17-4cedda32d7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.587435 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be91ca-9744-4e53-af17-4cedda32d7fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.588168 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be91ca-9744-4e53-af17-4cedda32d7fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.589899 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be91ca-9744-4e53-af17-4cedda32d7fb admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.601275 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 781/1577] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:39.605219 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be91ca-9744-4e53-af17-4cedda32d7fb admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.614205 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be91ca-9744-4e53-af17-4cedda32d7fb admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.617994 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 797/1578] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:04:39 2022] POST /identity/v3/projects => generated 393 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:39.620227 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520037c-ebcc-4ebb-975b-e3e44cbb993f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.621431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520037c-ebcc-4ebb-975b-e3e44cbb993f admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.621917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520037c-ebcc-4ebb-975b-e3e44cbb993f admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.622446 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520037c-ebcc-4ebb-975b-e3e44cbb993f admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.624019 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520037c-ebcc-4ebb-975b-e3e44cbb993f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.624501 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94bc126-881f-4de9-a356-aa5ef6ca4c24 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.627646 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520037c-ebcc-4ebb-975b-e3e44cbb993f admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.658672 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 782/1579] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:04:39 2022] POST /identity/v3/users => generated 285 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:39.659556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520037c-ebcc-4ebb-975b-e3e44cbb993f admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.669647 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520037c-ebcc-4ebb-975b-e3e44cbb993f admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.702876 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94bc126-881f-4de9-a356-aa5ef6ca4c24 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.704232 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:39.708812 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd27273-f043-4045-800b-81c4730664e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.709506 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd27273-f043-4045-800b-81c4730664e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.709794 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd27273-f043-4045-800b-81c4730664e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.710070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd27273-f043-4045-800b-81c4730664e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.710883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd27273-f043-4045-800b-81c4730664e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.712570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd27273-f043-4045-800b-81c4730664e0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.725514 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 798/1580] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:39.726450 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd27273-f043-4045-800b-81c4730664e0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.737279 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd27273-f043-4045-800b-81c4730664e0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.742286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d4518-8a1a-4ca0-b7d4-7838603f5dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.742883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d4518-8a1a-4ca0-b7d4-7838603f5dec admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.743164 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d4518-8a1a-4ca0-b7d4-7838603f5dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.743442 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d4518-8a1a-4ca0-b7d4-7838603f5dec admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.744170 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d4518-8a1a-4ca0-b7d4-7838603f5dec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.745819 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d4518-8a1a-4ca0-b7d4-7838603f5dec admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.761505 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 799/1581] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:39.761505 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d4518-8a1a-4ca0-b7d4-7838603f5dec admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.761877 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d6bae-0c2c-4f5c-b86c-52899d6bfa9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.761877 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d6bae-0c2c-4f5c-b86c-52899d6bfa9a admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.761877 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d6bae-0c2c-4f5c-b86c-52899d6bfa9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.761877 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d6bae-0c2c-4f5c-b86c-52899d6bfa9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.763081 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d6bae-0c2c-4f5c-b86c-52899d6bfa9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.766929 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d6bae-0c2c-4f5c-b86c-52899d6bfa9a admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.770119 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d4518-8a1a-4ca0-b7d4-7838603f5dec admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.774923 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef8e50-e073-441f-bff6-c9bed329c4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.775605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef8e50-e073-441f-bff6-c9bed329c4c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.775885 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef8e50-e073-441f-bff6-c9bed329c4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.776159 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef8e50-e073-441f-bff6-c9bed329c4c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.776859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef8e50-e073-441f-bff6-c9bed329c4c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.778910 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef8e50-e073-441f-bff6-c9bed329c4c8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.792659 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 783/1582] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:39.792659 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d6bae-0c2c-4f5c-b86c-52899d6bfa9a admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.793537 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:39.808537 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 800/1583] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:04:39 2022] POST /identity/v3/users => generated 287 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:39.809859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef8e50-e073-441f-bff6-c9bed329c4c8 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.811538 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d6bae-0c2c-4f5c-b86c-52899d6bfa9a admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.820015 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d622ff6-c37b-48ce-8d2f-902dd7451b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.821201 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d622ff6-c37b-48ce-8d2f-902dd7451b78 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.821722 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d622ff6-c37b-48ce-8d2f-902dd7451b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.822253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d622ff6-c37b-48ce-8d2f-902dd7451b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.822643 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef8e50-e073-441f-bff6-c9bed329c4c8 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.823555 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d622ff6-c37b-48ce-8d2f-902dd7451b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.826710 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d622ff6-c37b-48ce-8d2f-902dd7451b78 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.827482 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eceabd-74a6-48e5-9b7a-e009bd8d83be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.828167 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eceabd-74a6-48e5-9b7a-e009bd8d83be admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.828449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eceabd-74a6-48e5-9b7a-e009bd8d83be admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.828723 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eceabd-74a6-48e5-9b7a-e009bd8d83be admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.830255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eceabd-74a6-48e5-9b7a-e009bd8d83be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.831895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eceabd-74a6-48e5-9b7a-e009bd8d83be admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.839859 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 784/1584] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:39.843875 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d622ff6-c37b-48ce-8d2f-902dd7451b78 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.845273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 801/1585] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:39.846815 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eceabd-74a6-48e5-9b7a-e009bd8d83be admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.857643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d622ff6-c37b-48ce-8d2f-902dd7451b78 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.857915 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eceabd-74a6-48e5-9b7a-e009bd8d83be admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.863245 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad9bbf8-1d08-42dc-a2a3-4b881ac9f9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.863764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d334def-376f-44aa-aed2-283044e75379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.863954 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9bbf8-1d08-42dc-a2a3-4b881ac9f9cb admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.864216 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9bbf8-1d08-42dc-a2a3-4b881ac9f9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.864439 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d334def-376f-44aa-aed2-283044e75379 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.864609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9bbf8-1d08-42dc-a2a3-4b881ac9f9cb admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.864760 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d334def-376f-44aa-aed2-283044e75379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.864987 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d334def-376f-44aa-aed2-283044e75379 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.865485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad9bbf8-1d08-42dc-a2a3-4b881ac9f9cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.865710 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d334def-376f-44aa-aed2-283044e75379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.867349 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d334def-376f-44aa-aed2-283044e75379 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.867544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad9bbf8-1d08-42dc-a2a3-4b881ac9f9cb admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.879006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 802/1586] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:39.880032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d334def-376f-44aa-aed2-283044e75379 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.889584 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 785/1587] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:04:39 2022] POST /identity/v3/users => generated 297 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:39.890389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d334def-376f-44aa-aed2-283044e75379 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.890591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad9bbf8-1d08-42dc-a2a3-4b881ac9f9cb admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.896707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae167a-9a48-4387-91fd-07f325792ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.897579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae167a-9a48-4387-91fd-07f325792ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.897982 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae167a-9a48-4387-91fd-07f325792ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.898402 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae167a-9a48-4387-91fd-07f325792ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.899241 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae167a-9a48-4387-91fd-07f325792ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.901033 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae167a-9a48-4387-91fd-07f325792ce8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.902774 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad9bbf8-1d08-42dc-a2a3-4b881ac9f9cb admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.914432 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 803/1588] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:39.915329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae167a-9a48-4387-91fd-07f325792ce8 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.918833 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2694cf-8980-4bc9-b54f-1e9530221962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.920847 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2694cf-8980-4bc9-b54f-1e9530221962 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.921360 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2694cf-8980-4bc9-b54f-1e9530221962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.921839 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2694cf-8980-4bc9-b54f-1e9530221962 admin admin] PATH_INFO: `/v3/projects/d225e79dd1164e5f92160737fdf84c31/users/ccb6bc41828d40d5aca91d1ae51c6a99/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.927237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae167a-9a48-4387-91fd-07f325792ce8 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.932424 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebf327-2bab-453a-8bb8-ae5694c72796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:39.933422 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebf327-2bab-453a-8bb8-ae5694c72796 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:39.933832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebf327-2bab-453a-8bb8-ae5694c72796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:39.934256 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebf327-2bab-453a-8bb8-ae5694c72796 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:39.935083 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebf327-2bab-453a-8bb8-ae5694c72796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.936814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebf327-2bab-453a-8bb8-ae5694c72796 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.947631 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2694cf-8980-4bc9-b54f-1e9530221962 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d225e79dd1164e5f92160737fdf84c31, user_id=ccb6bc41828d40d5aca91d1ae51c6a99, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:39.948709 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: WARNING py.warnings [None req-db2694cf-8980-4bc9-b54f-1e9530221962 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 18:04:39.948709 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: warnings.warn(msg) May 26 18:04:39.948709 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: May 26 18:04:39.949031 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 804/1589] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:39.950701 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2694cf-8980-4bc9-b54f-1e9530221962 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:39.951230 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebf327-2bab-453a-8bb8-ae5694c72796 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:39.962429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebf327-2bab-453a-8bb8-ae5694c72796 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:39.964506 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 786/1590] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:04:39 2022] PUT /identity/v3/projects/d225e79dd1164e5f92160737fdf84c31/users/ccb6bc41828d40d5aca91d1ae51c6a99/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:39.965461 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2694cf-8980-4bc9-b54f-1e9530221962 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:40.029176 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff531e-7d66-4bf9-be2a-ef9a27eabd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:40.029856 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff531e-7d66-4bf9-be2a-ef9a27eabd6e admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.030145 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff531e-7d66-4bf9-be2a-ef9a27eabd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.030729 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff531e-7d66-4bf9-be2a-ef9a27eabd6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.031456 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff531e-7d66-4bf9-be2a-ef9a27eabd6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:40.034019 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff531e-7d66-4bf9-be2a-ef9a27eabd6e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:40.043368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2694cf-8980-4bc9-b54f-1e9530221962 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:40.050325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93ada5-9f45-48e2-a6ee-b66addba5517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:40.050576 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 805/1591] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:40.050683 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93ada5-9f45-48e2-a6ee-b66addba5517 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.050683 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93ada5-9f45-48e2-a6ee-b66addba5517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.050683 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93ada5-9f45-48e2-a6ee-b66addba5517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.051304 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff531e-7d66-4bf9-be2a-ef9a27eabd6e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:40.051548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e93ada5-9f45-48e2-a6ee-b66addba5517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:40.053284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e93ada5-9f45-48e2-a6ee-b66addba5517 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:40.068455 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 787/1592] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:40.068455 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93ada5-9f45-48e2-a6ee-b66addba5517 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:40.069938 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff531e-7d66-4bf9-be2a-ef9a27eabd6e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:40.074786 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2c49b-3f17-4930-b7e9-ccc4eb92eb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:40.075688 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2c49b-3f17-4930-b7e9-ccc4eb92eb4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.075975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2c49b-3f17-4930-b7e9-ccc4eb92eb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.076255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2c49b-3f17-4930-b7e9-ccc4eb92eb4d admin admin] PATH_INFO: `/v3/projects/eac4a6151fbb44d1b0b61636a97f56b6/users/dfbe23f3a0294148a88f4758d8470665/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.086338 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93ada5-9f45-48e2-a6ee-b66addba5517 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:40.094082 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2c49b-3f17-4930-b7e9-ccc4eb92eb4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eac4a6151fbb44d1b0b61636a97f56b6, user_id=dfbe23f3a0294148a88f4758d8470665, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:40.094387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1182f2-0314-4a2b-a688-dd4c8f5b0fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:40.096058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1182f2-0314-4a2b-a688-dd4c8f5b0fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.096560 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1182f2-0314-4a2b-a688-dd4c8f5b0fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.097046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1182f2-0314-4a2b-a688-dd4c8f5b0fa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.097660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2c49b-3f17-4930-b7e9-ccc4eb92eb4d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:40.098512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1182f2-0314-4a2b-a688-dd4c8f5b0fa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:40.101683 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1182f2-0314-4a2b-a688-dd4c8f5b0fa7 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:40.113929 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 806/1593] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:04:40 2022] PUT /identity/v3/projects/eac4a6151fbb44d1b0b61636a97f56b6/users/dfbe23f3a0294148a88f4758d8470665/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:40.118044 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2c49b-3f17-4930-b7e9-ccc4eb92eb4d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:40.119021 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 788/1594] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:40.120013 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1182f2-0314-4a2b-a688-dd4c8f5b0fa7 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:40.133303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1182f2-0314-4a2b-a688-dd4c8f5b0fa7 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:40.199194 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2c49b-3f17-4930-b7e9-ccc4eb92eb4d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:40.200978 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118985b5-0e3b-407d-95cd-f97691aceaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:40.202033 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118985b5-0e3b-407d-95cd-f97691aceaa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.202418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118985b5-0e3b-407d-95cd-f97691aceaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.202717 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118985b5-0e3b-407d-95cd-f97691aceaa8 admin admin] PATH_INFO: `/v3/projects/c0b976166b884524b9a3855712be72de/users/f6b5113ea0f4484ba91a5f0e6184ca35/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.204635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd7e689-f13b-4ba5-94f8-4fed44898e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:40.205360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7e689-f13b-4ba5-94f8-4fed44898e1a admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.205610 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7e689-f13b-4ba5-94f8-4fed44898e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.205880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7e689-f13b-4ba5-94f8-4fed44898e1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.206141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:40.206649 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd7e689-f13b-4ba5-94f8-4fed44898e1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:40.208367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd7e689-f13b-4ba5-94f8-4fed44898e1a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:40.219552 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 807/1595] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:40.221293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd7e689-f13b-4ba5-94f8-4fed44898e1a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:40.229375 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118985b5-0e3b-407d-95cd-f97691aceaa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0b976166b884524b9a3855712be72de, user_id=f6b5113ea0f4484ba91a5f0e6184ca35, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:40.232325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118985b5-0e3b-407d-95cd-f97691aceaa8 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:40.233750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd7e689-f13b-4ba5-94f8-4fed44898e1a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:40.240538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91892f0b-db17-41b4-8c27-0a38319e6829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:40.242122 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91892f0b-db17-41b4-8c27-0a38319e6829 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.242547 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91892f0b-db17-41b4-8c27-0a38319e6829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.242831 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91892f0b-db17-41b4-8c27-0a38319e6829 admin admin] PATH_INFO: `/v3/projects/9c09ef491df74d48b81e8346d6ee228a/users/5389ef3de3094a6ba85f928312c99dde/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.246938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 789/1596] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:04:40 2022] PUT /identity/v3/projects/c0b976166b884524b9a3855712be72de/users/f6b5113ea0f4484ba91a5f0e6184ca35/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:40.248250 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13673afe-1edc-4ecd-829c-28802c2298d0 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.248559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13673afe-1edc-4ecd-829c-28802c2298d0 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.248832 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13673afe-1edc-4ecd-829c-28802c2298d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.263835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91892f0b-db17-41b4-8c27-0a38319e6829 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c09ef491df74d48b81e8346d6ee228a, user_id=5389ef3de3094a6ba85f928312c99dde, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:40.268774 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91892f0b-db17-41b4-8c27-0a38319e6829 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:40.270949 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:40.279615 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-13673afe-1edc-4ecd-829c-28802c2298d0 None None] MFA Rules not processed for user `ccb6bc41828d40d5aca91d1ae51c6a99`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:40.281093 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-13673afe-1edc-4ecd-829c-28802c2298d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:40.285082 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 808/1597] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:04:40 2022] PUT /identity/v3/projects/9c09ef491df74d48b81e8346d6ee228a/users/5389ef3de3094a6ba85f928312c99dde/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:40.286030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91892f0b-db17-41b4-8c27-0a38319e6829 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:40.353346 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 790/1598] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:40.354089 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13673afe-1edc-4ecd-829c-28802c2298d0 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:40.372872 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91892f0b-db17-41b4-8c27-0a38319e6829 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:40.379463 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb719b42-8964-4227-82b5-d8901e24ce7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:40.380270 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb719b42-8964-4227-82b5-d8901e24ce7b admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.380560 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb719b42-8964-4227-82b5-d8901e24ce7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.380874 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb719b42-8964-4227-82b5-d8901e24ce7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.381877 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb719b42-8964-4227-82b5-d8901e24ce7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:40.383948 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb719b42-8964-4227-82b5-d8901e24ce7b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:40.398607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 809/1599] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:40.400682 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcb80e-d6d4-45c3-8a3a-4b08774db07f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.401070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcb80e-d6d4-45c3-8a3a-4b08774db07f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.401420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcb80e-d6d4-45c3-8a3a-4b08774db07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.430933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1fbcb80e-d6d4-45c3-8a3a-4b08774db07f None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:40.432417 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1fbcb80e-d6d4-45c3-8a3a-4b08774db07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:40.432863 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13673afe-1edc-4ecd-829c-28802c2298d0 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:40.437517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd32f8-e074-4399-af5f-d9aeb371e933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:40.439007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd32f8-e074-4399-af5f-d9aeb371e933 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.439297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd32f8-e074-4399-af5f-d9aeb371e933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.439578 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd32f8-e074-4399-af5f-d9aeb371e933 admin admin] PATH_INFO: `/v3/projects/6b746bd093c749c5a2ffe6fd3d3e0417/users/81b8f193d3354e9784dc62bbae040e72/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.447257 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 810/1600] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:40.448164 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbcb80e-d6d4-45c3-8a3a-4b08774db07f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:40.459198 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbcb80e-d6d4-45c3-8a3a-4b08774db07f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:40.464142 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77209bdc-2f69-4921-9feb-751f4db707d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:40.465072 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77209bdc-2f69-4921-9feb-751f4db707d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.465189 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd32f8-e074-4399-af5f-d9aeb371e933 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b746bd093c749c5a2ffe6fd3d3e0417, user_id=81b8f193d3354e9784dc62bbae040e72, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:40.465387 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77209bdc-2f69-4921-9feb-751f4db707d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.465661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77209bdc-2f69-4921-9feb-751f4db707d4 admin admin] PATH_INFO: `/v3/projects/59bf92ddd0db48cb81c512a9a2b5b6cf/users/ac72f0d5232f492da69421440a89cdea/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.469853 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd32f8-e074-4399-af5f-d9aeb371e933 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:40.485154 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77209bdc-2f69-4921-9feb-751f4db707d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59bf92ddd0db48cb81c512a9a2b5b6cf, user_id=ac72f0d5232f492da69421440a89cdea, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:40.490756 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 791/1601] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:04:40 2022] PUT /identity/v3/projects/6b746bd093c749c5a2ffe6fd3d3e0417/users/81b8f193d3354e9784dc62bbae040e72/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:40.490756 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd32f8-e074-4399-af5f-d9aeb371e933 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:40.491065 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77209bdc-2f69-4921-9feb-751f4db707d4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:40.522587 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 811/1602] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:04:40 2022] PUT /identity/v3/projects/59bf92ddd0db48cb81c512a9a2b5b6cf/users/ac72f0d5232f492da69421440a89cdea/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:40.527960 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e45d57-da40-4401-bbb5-82c3729737ee None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.528638 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e45d57-da40-4401-bbb5-82c3729737ee None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.529269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e45d57-da40-4401-bbb5-82c3729737ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.551451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-76e45d57-da40-4401-bbb5-82c3729737ee None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:40.552844 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-76e45d57-da40-4401-bbb5-82c3729737ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:40.569878 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:40.583133 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd32f8-e074-4399-af5f-d9aeb371e933 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:40.639916 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 812/1603] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:40.644929 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42487e1d-fbc8-4424-ad97-1e74e5f95f34 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.645408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42487e1d-fbc8-4424-ad97-1e74e5f95f34 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.645877 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42487e1d-fbc8-4424-ad97-1e74e5f95f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.673975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb65d0-c27a-4822-8bf0-62886d9a03e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:40.674742 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb65d0-c27a-4822-8bf0-62886d9a03e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.675030 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb65d0-c27a-4822-8bf0-62886d9a03e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.675368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb65d0-c27a-4822-8bf0-62886d9a03e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.676350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb65d0-c27a-4822-8bf0-62886d9a03e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:40.678386 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb65d0-c27a-4822-8bf0-62886d9a03e1 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:40.687249 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-42487e1d-fbc8-4424-ad97-1e74e5f95f34 None None] MFA Rules not processed for user `5389ef3de3094a6ba85f928312c99dde`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:40.688799 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-42487e1d-fbc8-4424-ad97-1e74e5f95f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:40.692899 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 792/1604] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:40.692899 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292b0cb-ecde-4e80-88b4-d88841665e37 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.692899 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292b0cb-ecde-4e80-88b4-d88841665e37 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.692899 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292b0cb-ecde-4e80-88b4-d88841665e37 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.693907 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 793/1605] 10.222.0.76 () {54 vars in 961 bytes} [Thu May 26 18:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:40.697017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292b0cb-ecde-4e80-88b4-d88841665e37 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:40.710890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292b0cb-ecde-4e80-88b4-d88841665e37 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:40.715587 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768cc1e-edab-4e92-b082-4e2e8cc97d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:40.716501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768cc1e-edab-4e92-b082-4e2e8cc97d5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.716800 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768cc1e-edab-4e92-b082-4e2e8cc97d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.717071 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768cc1e-edab-4e92-b082-4e2e8cc97d5c admin admin] PATH_INFO: `/v3/projects/1ff84f5142c24aa4a50fbcb6510a5624/users/6d54cc1979ca43b7818bd0d21c1d735f/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.737679 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768cc1e-edab-4e92-b082-4e2e8cc97d5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ff84f5142c24aa4a50fbcb6510a5624, user_id=6d54cc1979ca43b7818bd0d21c1d735f, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:40.740400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768cc1e-edab-4e92-b082-4e2e8cc97d5c admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:40.748779 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:40.756181 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 794/1606] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:04:40 2022] PUT /identity/v3/projects/1ff84f5142c24aa4a50fbcb6510a5624/users/6d54cc1979ca43b7818bd0d21c1d735f/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:40.756999 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768cc1e-edab-4e92-b082-4e2e8cc97d5c admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:40.777699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 813/1607] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:40.779563 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7c8bf-d307-4c4d-a08d-f8b58d2d3949 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.779939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7c8bf-d307-4c4d-a08d-f8b58d2d3949 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.780551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7c8bf-d307-4c4d-a08d-f8b58d2d3949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.806019 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8cf7c8bf-d307-4c4d-a08d-f8b58d2d3949 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:40.807529 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8cf7c8bf-d307-4c4d-a08d-f8b58d2d3949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:40.847421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768cc1e-edab-4e92-b082-4e2e8cc97d5c admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:40.852755 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a188cc0-472f-40ca-a577-5af061c56fa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:40.853420 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a188cc0-472f-40ca-a577-5af061c56fa1 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.853703 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a188cc0-472f-40ca-a577-5af061c56fa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.853979 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a188cc0-472f-40ca-a577-5af061c56fa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.888237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 814/1608] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:40.893826 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213b8b7-032f-4aec-b554-0c2ce6a2e683 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.894297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213b8b7-032f-4aec-b554-0c2ce6a2e683 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.894567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213b8b7-032f-4aec-b554-0c2ce6a2e683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:40.929554 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0213b8b7-032f-4aec-b554-0c2ce6a2e683 None None] MFA Rules not processed for user `ac72f0d5232f492da69421440a89cdea`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:40.931314 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0213b8b7-032f-4aec-b554-0c2ce6a2e683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:40.955461 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a188cc0-472f-40ca-a577-5af061c56fa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:40.956968 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a188cc0-472f-40ca-a577-5af061c56fa1 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:40.968166 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 795/1609] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:04:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:40.969142 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a188cc0-472f-40ca-a577-5af061c56fa1 service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:40.978896 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a188cc0-472f-40ca-a577-5af061c56fa1 service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:40.983584 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff02edc-37d3-4936-ad60-42712217c8b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:40.984388 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff02edc-37d3-4936-ad60-42712217c8b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:40.984568 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff02edc-37d3-4936-ad60-42712217c8b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:40.984858 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff02edc-37d3-4936-ad60-42712217c8b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.000642 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff02edc-37d3-4936-ad60-42712217c8b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:41.003208 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff02edc-37d3-4936-ad60-42712217c8b6 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:41.015942 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 796/1610] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:04:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:41.017543 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff02edc-37d3-4936-ad60-42712217c8b6 service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:41.029303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff02edc-37d3-4936-ad60-42712217c8b6 service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:41.034584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 815/1611] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:40 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:41.037115 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4195b49-0e7e-4afc-921a-51fde71c360e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.037472 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4195b49-0e7e-4afc-921a-51fde71c360e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.038019 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4195b49-0e7e-4afc-921a-51fde71c360e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.039486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4b446-af92-41c8-9209-a62aeeec15dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:41.040738 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4b446-af92-41c8-9209-a62aeeec15dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.040738 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4b446-af92-41c8-9209-a62aeeec15dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.040976 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4b446-af92-41c8-9209-a62aeeec15dd admin admin] PATH_INFO: `/v3/projects/8c49a9a21f1346a0bc8d774afc7951a2/users/61f6dbd4328e459eb7d4f029551648bb/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.056036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:41.069663 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4b446-af92-41c8-9209-a62aeeec15dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c49a9a21f1346a0bc8d774afc7951a2, user_id=61f6dbd4328e459eb7d4f029551648bb, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:41.072455 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4b446-af92-41c8-9209-a62aeeec15dd admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:41.073025 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b4195b49-0e7e-4afc-921a-51fde71c360e None None] MFA Rules not processed for user `1899c317083d40f5b490cd9eb63fe521`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:41.076295 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b4195b49-0e7e-4afc-921a-51fde71c360e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:41.093110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 797/1612] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:04:41 2022] PUT /identity/v3/projects/8c49a9a21f1346a0bc8d774afc7951a2/users/61f6dbd4328e459eb7d4f029551648bb/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:41.100577 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7e86c-a1e1-41b9-a272-843dca4f0750 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.102125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7e86c-a1e1-41b9-a272-843dca4f0750 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.102947 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7e86c-a1e1-41b9-a272-843dca4f0750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.162968 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-dfb7e86c-a1e1-41b9-a272-843dca4f0750 None None] MFA Rules not processed for user `6d54cc1979ca43b7818bd0d21c1d735f`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:41.165164 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-dfb7e86c-a1e1-41b9-a272-843dca4f0750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:41.201003 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 816/1613] 10.222.0.76 () {58 vars in 1058 bytes} [Thu May 26 18:04:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:41.202506 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eace72-91af-48e7-86d9-84df52e26d5e None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.202976 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eace72-91af-48e7-86d9-84df52e26d5e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.203260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eace72-91af-48e7-86d9-84df52e26d5e None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.205675 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 817/1614] 10.222.0.76 () {54 vars in 961 bytes} [Thu May 26 18:04:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:41.206476 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eace72-91af-48e7-86d9-84df52e26d5e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:41.232334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:41.276792 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 798/1615] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:41.277745 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7e86c-a1e1-41b9-a272-843dca4f0750 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:41.290952 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eace72-91af-48e7-86d9-84df52e26d5e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:41.295868 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71ca50-561c-4686-87af-534e9cfde307 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:41.296591 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71ca50-561c-4686-87af-534e9cfde307 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.296883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71ca50-561c-4686-87af-534e9cfde307 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.297328 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71ca50-561c-4686-87af-534e9cfde307 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.368219 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7e86c-a1e1-41b9-a272-843dca4f0750 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:41.372972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2bb788-1c52-49cf-8099-4c08a17f49a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:41.373757 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2bb788-1c52-49cf-8099-4c08a17f49a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.374311 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2bb788-1c52-49cf-8099-4c08a17f49a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.374682 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2bb788-1c52-49cf-8099-4c08a17f49a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.375484 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2bb788-1c52-49cf-8099-4c08a17f49a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:41.378459 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2bb788-1c52-49cf-8099-4c08a17f49a5 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:41.390113 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71ca50-561c-4686-87af-534e9cfde307 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:41.392759 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71ca50-561c-4686-87af-534e9cfde307 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:41.410071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 818/1616] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:04:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:41.412015 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b473e07-69d5-454a-85f8-0e91d655b729 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.412406 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b473e07-69d5-454a-85f8-0e91d655b729 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.412819 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b473e07-69d5-454a-85f8-0e91d655b729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.415115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 799/1617] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:04:41 2022] POST /identity/v3/projects => generated 365 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:41.416664 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2bb788-1c52-49cf-8099-4c08a17f49a5 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:41.436030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1b473e07-69d5-454a-85f8-0e91d655b729 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:41.437391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1b473e07-69d5-454a-85f8-0e91d655b729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:41.501733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:41.531945 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2bb788-1c52-49cf-8099-4c08a17f49a5 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:41.537737 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 819/1618] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:41.539423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc8c81-2225-4e26-a0a6-4da5e2098ad5 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.539824 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc8c81-2225-4e26-a0a6-4da5e2098ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.540212 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc8c81-2225-4e26-a0a6-4da5e2098ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.549582 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda2728-679c-43e5-a984-ef0f1bdd239a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:41.551366 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda2728-679c-43e5-a984-ef0f1bdd239a service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.552224 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda2728-679c-43e5-a984-ef0f1bdd239a service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.553113 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda2728-679c-43e5-a984-ef0f1bdd239a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.566941 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-62bc8c81-2225-4e26-a0a6-4da5e2098ad5 None None] MFA Rules not processed for user `1899c317083d40f5b490cd9eb63fe521`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:41.568725 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-62bc8c81-2225-4e26-a0a6-4da5e2098ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:41.579672 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 820/1619] 10.222.0.76 () {58 vars in 1058 bytes} [Thu May 26 18:04:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:41.581010 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc8c81-2225-4e26-a0a6-4da5e2098ad5 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:41.596274 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc8c81-2225-4e26-a0a6-4da5e2098ad5 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:41.603478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac986b9c-36d0-4e37-9cc0-3c581f7d7be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:41.604338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac986b9c-36d0-4e37-9cc0-3c581f7d7be5 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.605454 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac986b9c-36d0-4e37-9cc0-3c581f7d7be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.605943 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac986b9c-36d0-4e37-9cc0-3c581f7d7be5 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.606933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac986b9c-36d0-4e37-9cc0-3c581f7d7be5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:41.609357 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac986b9c-36d0-4e37-9cc0-3c581f7d7be5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:41.640389 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda2728-679c-43e5-a984-ef0f1bdd239a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:41.641750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 821/1620] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:04:41 2022] POST /identity/v3/users => generated 289 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:41.644246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac986b9c-36d0-4e37-9cc0-3c581f7d7be5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:41.665562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda2728-679c-43e5-a984-ef0f1bdd239a service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:41.694231 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 800/1621] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:04:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:41.695303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda2728-679c-43e5-a984-ef0f1bdd239a service nova] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:41.712379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:41.729071 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda2728-679c-43e5-a984-ef0f1bdd239a service nova] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:41.730316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac986b9c-36d0-4e37-9cc0-3c581f7d7be5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:41.738414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a4819-6528-412a-beaa-dbb62b1ea65a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:41.739361 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a4819-6528-412a-beaa-dbb62b1ea65a service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.740032 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a4819-6528-412a-beaa-dbb62b1ea65a service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.740353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a4819-6528-412a-beaa-dbb62b1ea65a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.744920 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30829716-c65d-45f2-9d5a-f7fda5be43f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:41.748086 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30829716-c65d-45f2-9d5a-f7fda5be43f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.748813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30829716-c65d-45f2-9d5a-f7fda5be43f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.752279 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30829716-c65d-45f2-9d5a-f7fda5be43f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.805151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30829716-c65d-45f2-9d5a-f7fda5be43f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:41.807378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30829716-c65d-45f2-9d5a-f7fda5be43f0 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:41.836857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 822/1622] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:04:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:41.837843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30829716-c65d-45f2-9d5a-f7fda5be43f0 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:41.863311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30829716-c65d-45f2-9d5a-f7fda5be43f0 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:41.876904 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d53aea-a3af-41d4-a3c3-2bd63c44f4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:41.879151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d53aea-a3af-41d4-a3c3-2bd63c44f4bb admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.879849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d53aea-a3af-41d4-a3c3-2bd63c44f4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.880146 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d53aea-a3af-41d4-a3c3-2bd63c44f4bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.882773 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d53aea-a3af-41d4-a3c3-2bd63c44f4bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:41.885697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d53aea-a3af-41d4-a3c3-2bd63c44f4bb admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:41.896949 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a4819-6528-412a-beaa-dbb62b1ea65a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:41.898706 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a4819-6528-412a-beaa-dbb62b1ea65a service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:41.913424 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 823/1623] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:41.919172 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d53aea-a3af-41d4-a3c3-2bd63c44f4bb admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:41.928470 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 801/1624] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:04:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:41.938125 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d53aea-a3af-41d4-a3c3-2bd63c44f4bb admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:41.953720 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04577e1c-e58c-498f-acfe-4efb841c1ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:41.956294 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04577e1c-e58c-498f-acfe-4efb841c1ec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:41.956653 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04577e1c-e58c-498f-acfe-4efb841c1ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:41.956951 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04577e1c-e58c-498f-acfe-4efb841c1ec0 admin admin] PATH_INFO: `/v3/projects/4d4d85df62fe46a9af5f2f5e4b7ab971/users/40b87a7069984a5bac4215179d3e33ae/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:41.999328 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04577e1c-e58c-498f-acfe-4efb841c1ec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d4d85df62fe46a9af5f2f5e4b7ab971, user_id=40b87a7069984a5bac4215179d3e33ae, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:42.003419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04577e1c-e58c-498f-acfe-4efb841c1ec0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:42.029240 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 824/1625] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:04:41 2022] PUT /identity/v3/projects/4d4d85df62fe46a9af5f2f5e4b7ab971/users/40b87a7069984a5bac4215179d3e33ae/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:42.034492 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a4819-6528-412a-beaa-dbb62b1ea65a service nova] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:42.053945 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a4819-6528-412a-beaa-dbb62b1ea65a service nova] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:42.145996 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04577e1c-e58c-498f-acfe-4efb841c1ec0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:42.182119 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185871b2-8361-4da5-8414-4aaab17b41da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:42.183184 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185871b2-8361-4da5-8414-4aaab17b41da admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:42.183799 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185871b2-8361-4da5-8414-4aaab17b41da admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:42.184297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185871b2-8361-4da5-8414-4aaab17b41da admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:42.185366 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185871b2-8361-4da5-8414-4aaab17b41da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:42.187518 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185871b2-8361-4da5-8414-4aaab17b41da admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:42.205195 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 802/1626] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:04:42.209515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185871b2-8361-4da5-8414-4aaab17b41da admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:42.233043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185871b2-8361-4da5-8414-4aaab17b41da admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:42.242807 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba631b-bab5-4baa-81e6-ee77cc8620c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:42.244578 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba631b-bab5-4baa-81e6-ee77cc8620c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:42.259460 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba631b-bab5-4baa-81e6-ee77cc8620c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:42.260202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba631b-bab5-4baa-81e6-ee77cc8620c5 admin admin] PATH_INFO: `/v3/projects/4d4d85df62fe46a9af5f2f5e4b7ab971/users/40b87a7069984a5bac4215179d3e33ae/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:42.265530 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba631b-bab5-4baa-81e6-ee77cc8620c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d4d85df62fe46a9af5f2f5e4b7ab971, user_id=40b87a7069984a5bac4215179d3e33ae, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:42.275517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba631b-bab5-4baa-81e6-ee77cc8620c5 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:42.306379 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 803/1627] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:04:42 2022] PUT /identity/v3/projects/4d4d85df62fe46a9af5f2f5e4b7ab971/users/40b87a7069984a5bac4215179d3e33ae/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:42.318089 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23536857-3e18-47cd-8a4c-3e1ca901fc3c None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:42.318980 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23536857-3e18-47cd-8a4c-3e1ca901fc3c None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:42.319548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23536857-3e18-47cd-8a4c-3e1ca901fc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:42.338364 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04577e1c-e58c-498f-acfe-4efb841c1ec0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:42.399147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-23536857-3e18-47cd-8a4c-3e1ca901fc3c None None] MFA Rules not processed for user `40b87a7069984a5bac4215179d3e33ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:42.400399 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-23536857-3e18-47cd-8a4c-3e1ca901fc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:42.421290 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:42.449320 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60893b-37af-442f-8e42-09e41913832e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:42.449994 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60893b-37af-442f-8e42-09e41913832e service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:42.450321 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60893b-37af-442f-8e42-09e41913832e service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:42.450596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60893b-37af-442f-8e42-09e41913832e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:42.545249 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:42.575548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 804/1628] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:42 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:04:42.577143 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576a44b-635a-4c01-895a-2629b3a141bb None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:42.577430 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576a44b-635a-4c01-895a-2629b3a141bb None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:42.577704 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576a44b-635a-4c01-895a-2629b3a141bb None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:42.580069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 805/1629] 10.222.0.76 () {54 vars in 912 bytes} [Thu May 26 18:04:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:42.581169 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576a44b-635a-4c01-895a-2629b3a141bb None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:42.592146 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f60893b-37af-442f-8e42-09e41913832e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:42.593860 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f60893b-37af-442f-8e42-09e41913832e service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:42.617465 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 825/1630] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:04:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:42.622539 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19381a48-072d-4f9c-a6d8-a813b216ad42 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:42.623201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19381a48-072d-4f9c-a6d8-a813b216ad42 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:42.623771 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19381a48-072d-4f9c-a6d8-a813b216ad42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:42.695467 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-19381a48-072d-4f9c-a6d8-a813b216ad42 None None] MFA Rules not processed for user `1899c317083d40f5b490cd9eb63fe521`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:42.698808 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-19381a48-072d-4f9c-a6d8-a813b216ad42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:42.702571 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576a44b-635a-4c01-895a-2629b3a141bb None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:42.711909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b020d52-ddbf-4844-9fa1-a995e553754c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:42.712590 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b020d52-ddbf-4844-9fa1-a995e553754c service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:42.712868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b020d52-ddbf-4844-9fa1-a995e553754c service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:42.713147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b020d52-ddbf-4844-9fa1-a995e553754c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:42.717261 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 826/1631] 10.222.0.76 () {58 vars in 1009 bytes} [Thu May 26 18:04:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:42.719005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b463ac-a3ca-4834-b3fa-2f06c92332a2 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:42.719400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b463ac-a3ca-4834-b3fa-2f06c92332a2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:42.720867 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b463ac-a3ca-4834-b3fa-2f06c92332a2 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:42.723094 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 827/1632] 10.222.0.76 () {54 vars in 912 bytes} [Thu May 26 18:04:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:42.730125 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ca449-24de-463c-b0a1-5c227bea9f5e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:42.730588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ca449-24de-463c-b0a1-5c227bea9f5e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:42.731332 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ca449-24de-463c-b0a1-5c227bea9f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:42.734702 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b020d52-ddbf-4844-9fa1-a995e553754c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:42.738333 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b020d52-ddbf-4844-9fa1-a995e553754c service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:42.754373 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-737ca449-24de-463c-b0a1-5c227bea9f5e None None] MFA Rules not processed for user `1899c317083d40f5b490cd9eb63fe521`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:42.755927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-737ca449-24de-463c-b0a1-5c227bea9f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:42.764334 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 806/1633] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:04:42 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:42.774028 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 828/1634] 10.222.0.76 () {58 vars in 1009 bytes} [Thu May 26 18:04:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:42.775518 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b020d52-ddbf-4844-9fa1-a995e553754c service cinder] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:42.797528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b020d52-ddbf-4844-9fa1-a995e553754c service cinder] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:42.805552 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc86e9-12ec-4406-ae43-1e8fce5b9ea7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:42.806411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc86e9-12ec-4406-ae43-1e8fce5b9ea7 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:42.807446 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc86e9-12ec-4406-ae43-1e8fce5b9ea7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:42.807863 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc86e9-12ec-4406-ae43-1e8fce5b9ea7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:42.837806 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc86e9-12ec-4406-ae43-1e8fce5b9ea7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:42.839387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc86e9-12ec-4406-ae43-1e8fce5b9ea7 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:42.845130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ca449-24de-463c-b0a1-5c227bea9f5e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:42.853821 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 807/1635] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:04:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:42.855327 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ca449-24de-463c-b0a1-5c227bea9f5e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:42.859619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53bded-760e-4286-a987-1ebe0b362130 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:42.861047 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53bded-760e-4286-a987-1ebe0b362130 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:42.861478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53bded-760e-4286-a987-1ebe0b362130 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:42.862072 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53bded-760e-4286-a987-1ebe0b362130 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:42.874409 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53bded-760e-4286-a987-1ebe0b362130 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:42.876107 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53bded-760e-4286-a987-1ebe0b362130 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:42.901151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 829/1636] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:04:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:04:43.532431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9978c-054b-4b0c-82b6-fe7e5ef4eadd None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:43.533100 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9978c-054b-4b0c-82b6-fe7e5ef4eadd None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:43.533522 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9978c-054b-4b0c-82b6-fe7e5ef4eadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:43.635430 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-a1f9978c-054b-4b0c-82b6-fe7e5ef4eadd None None] MFA Rules not processed for user `81b8f193d3354e9784dc62bbae040e72`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:43.636707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-a1f9978c-054b-4b0c-82b6-fe7e5ef4eadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:43.850211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 808/1637] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:04:43.864052 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ed7b7-3b08-4e2b-8be7-49b98a01c16d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:43.864521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ed7b7-3b08-4e2b-8be7-49b98a01c16d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:43.864993 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ed7b7-3b08-4e2b-8be7-49b98a01c16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:43.903400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e32ed7b7-3b08-4e2b-8be7-49b98a01c16d None None] MFA Rules not processed for user `81b8f193d3354e9784dc62bbae040e72`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:43.904615 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e32ed7b7-3b08-4e2b-8be7-49b98a01c16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:43.929445 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 830/1638] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:44.133246 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9978c-054b-4b0c-82b6-fe7e5ef4eadd None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:44.161304 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9978c-054b-4b0c-82b6-fe7e5ef4eadd None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:44.180514 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ef94d-f336-477a-bf75-898261707678 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:44.181379 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ef94d-f336-477a-bf75-898261707678 service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:44.181845 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ef94d-f336-477a-bf75-898261707678 service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:44.182310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ef94d-f336-477a-bf75-898261707678 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:44.197856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ef94d-f336-477a-bf75-898261707678 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:44.199911 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ef94d-f336-477a-bf75-898261707678 service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:44.224414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 809/1639] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:04:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:44.808015 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ed7b7-3b08-4e2b-8be7-49b98a01c16d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:45.015833 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ed7b7-3b08-4e2b-8be7-49b98a01c16d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:45.027803 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881ef14-47ac-48ad-b727-a52bbee55579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:45.029473 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881ef14-47ac-48ad-b727-a52bbee55579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:45.029895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881ef14-47ac-48ad-b727-a52bbee55579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:45.030575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881ef14-47ac-48ad-b727-a52bbee55579 admin admin] PATH_INFO: `/v3/users/ccb6bc41828d40d5aca91d1ae51c6a99` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:45.033854 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881ef14-47ac-48ad-b727-a52bbee55579 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccb6bc41828d40d5aca91d1ae51c6a99)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:45.037841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881ef14-47ac-48ad-b727-a52bbee55579 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:45.131304 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d881ef14-47ac-48ad-b727-a52bbee55579 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccb6bc41828d40d5aca91d1ae51c6a99'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:45.152515 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d881ef14-47ac-48ad-b727-a52bbee55579 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccb6bc41828d40d5aca91d1ae51c6a99'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:45.196567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d881ef14-47ac-48ad-b727-a52bbee55579 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccb6bc41828d40d5aca91d1ae51c6a99'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:45.197455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d881ef14-47ac-48ad-b727-a52bbee55579 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccb6bc41828d40d5aca91d1ae51c6a99'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:45.198329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d881ef14-47ac-48ad-b727-a52bbee55579 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccb6bc41828d40d5aca91d1ae51c6a99'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:45.246158 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 831/1640] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:04:44 2022] DELETE /identity/v3/users/ccb6bc41828d40d5aca91d1ae51c6a99 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:45.252339 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93365ec-e172-4705-ba43-24426a8e76a7 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:45.252917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93365ec-e172-4705-ba43-24426a8e76a7 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:45.253324 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93365ec-e172-4705-ba43-24426a8e76a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:45.327942 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce148c1-6fcb-4359-8369-e1450240b8c4 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:45.328334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce148c1-6fcb-4359-8369-e1450240b8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:45.328742 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce148c1-6fcb-4359-8369-e1450240b8c4 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:45.331819 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 832/1641] 10.222.0.76 () {54 vars in 912 bytes} [Thu May 26 18:04:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:04:45.358224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6685e4-0dfc-47ac-b4a7-1a457b305825 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:45.358576 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6685e4-0dfc-47ac-b4a7-1a457b305825 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:45.358981 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6685e4-0dfc-47ac-b4a7-1a457b305825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:45.363561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-a93365ec-e172-4705-ba43-24426a8e76a7 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:45.366979 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-a93365ec-e172-4705-ba43-24426a8e76a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:45.391765 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:45.479335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5f6685e4-0dfc-47ac-b4a7-1a457b305825 None None] MFA Rules not processed for user `8df66990a3734209a350c8b6768d259f`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:45.509438 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5f6685e4-0dfc-47ac-b4a7-1a457b305825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:45.553084 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:45.598486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 810/1642] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:45.600077 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93365ec-e172-4705-ba43-24426a8e76a7 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:45.625601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-a93365ec-e172-4705-ba43-24426a8e76a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:45.772564 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 833/1643] 10.222.0.76 () {58 vars in 1009 bytes} [Thu May 26 18:04:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:45.778150 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6685e4-0dfc-47ac-b4a7-1a457b305825 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:45.778362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93365ec-e172-4705-ba43-24426a8e76a7 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:45.792231 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a807a-0499-4a81-8174-37eb0175c2b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:45.793313 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a807a-0499-4a81-8174-37eb0175c2b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:45.793413 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6685e4-0dfc-47ac-b4a7-1a457b305825 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:45.794415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a807a-0499-4a81-8174-37eb0175c2b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:45.794988 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a807a-0499-4a81-8174-37eb0175c2b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:45.804069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c602f38-917a-4d2d-8687-84b826969667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:45.804755 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c602f38-917a-4d2d-8687-84b826969667 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:45.805041 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c602f38-917a-4d2d-8687-84b826969667 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:45.805319 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c602f38-917a-4d2d-8687-84b826969667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:45.814920 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c602f38-917a-4d2d-8687-84b826969667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:45.817152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c602f38-917a-4d2d-8687-84b826969667 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:45.828223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a807a-0499-4a81-8174-37eb0175c2b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:45.832571 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a807a-0499-4a81-8174-37eb0175c2b3 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:45.833762 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 834/1644] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:04:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:04:45.848520 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 811/1645] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:04:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:47.075991 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c602f38-917a-4d2d-8687-84b826969667 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:47.078228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2c602f38-917a-4d2d-8687-84b826969667 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:47.104331 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c602f38-917a-4d2d-8687-84b826969667 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:47.114324 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04448bb-c643-4693-9bd7-314c3af08c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:47.115477 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04448bb-c643-4693-9bd7-314c3af08c3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:47.115965 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04448bb-c643-4693-9bd7-314c3af08c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:47.116564 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04448bb-c643-4693-9bd7-314c3af08c3d admin admin] PATH_INFO: `/v3/users/ac72f0d5232f492da69421440a89cdea` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:47.118369 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04448bb-c643-4693-9bd7-314c3af08c3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac72f0d5232f492da69421440a89cdea)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:47.120454 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04448bb-c643-4693-9bd7-314c3af08c3d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:47.299121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b04448bb-c643-4693-9bd7-314c3af08c3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac72f0d5232f492da69421440a89cdea'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:47.311375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b04448bb-c643-4693-9bd7-314c3af08c3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac72f0d5232f492da69421440a89cdea'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:47.335702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b04448bb-c643-4693-9bd7-314c3af08c3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac72f0d5232f492da69421440a89cdea'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:47.337448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b04448bb-c643-4693-9bd7-314c3af08c3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac72f0d5232f492da69421440a89cdea'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:47.338221 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b04448bb-c643-4693-9bd7-314c3af08c3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac72f0d5232f492da69421440a89cdea'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:47.358846 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b79d3-41f4-4fe7-a64e-484ede09c56a None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:47.359935 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b79d3-41f4-4fe7-a64e-484ede09c56a None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:47.360405 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b79d3-41f4-4fe7-a64e-484ede09c56a None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:47.362491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 812/1646] 10.222.0.76 () {54 vars in 960 bytes} [Thu May 26 18:04:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:04:47.363242 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 835/1647] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:04:47 2022] DELETE /identity/v3/users/ac72f0d5232f492da69421440a89cdea => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:47.367424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334ba33-0fc0-489f-a6a4-c5a9f75effd5 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:47.367914 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334ba33-0fc0-489f-a6a4-c5a9f75effd5 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:47.368350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334ba33-0fc0-489f-a6a4-c5a9f75effd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:47.374366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52afadd-3221-485f-912d-205bec48136e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:47.374856 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52afadd-3221-485f-912d-205bec48136e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:47.375263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52afadd-3221-485f-912d-205bec48136e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:47.412089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e52afadd-3221-485f-912d-205bec48136e None None] MFA Rules not processed for user `596f126382e3440fb3cf7c357eef889f`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:47.413616 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e52afadd-3221-485f-912d-205bec48136e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:47.417211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-a334ba33-0fc0-489f-a6a4-c5a9f75effd5 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:47.430004 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-a334ba33-0fc0-489f-a6a4-c5a9f75effd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:47.511959 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 836/1648] 10.222.0.76 () {58 vars in 1057 bytes} [Thu May 26 18:04:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) May 26 18:04:47.523069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52afadd-3221-485f-912d-205bec48136e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:47.545487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52afadd-3221-485f-912d-205bec48136e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:47.554620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd6630-4b32-4783-b4cd-efe118ddc12c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '596f126382e3440fb3cf7c357eef889f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:47.555571 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd6630-4b32-4783-b4cd-efe118ddc12c service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:47.555853 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd6630-4b32-4783-b4cd-efe118ddc12c service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:47.556277 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd6630-4b32-4783-b4cd-efe118ddc12c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:47.565374 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6bfd6630-4b32-4783-b4cd-efe118ddc12c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:47.595632 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd6630-4b32-4783-b4cd-efe118ddc12c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:47.596730 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 813/1649] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:47.599786 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd6630-4b32-4783-b4cd-efe118ddc12c service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:47.632253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 837/1650] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:04:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:47.681260 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334ba33-0fc0-489f-a6a4-c5a9f75effd5 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:47.683851 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-a334ba33-0fc0-489f-a6a4-c5a9f75effd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:47.890669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd6630-4b32-4783-b4cd-efe118ddc12c service placement] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:47.892999 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6bfd6630-4b32-4783-b4cd-efe118ddc12c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:47.927327 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334ba33-0fc0-489f-a6a4-c5a9f75effd5 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:47.938125 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd6630-4b32-4783-b4cd-efe118ddc12c service placement] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:47.950427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b353b6d-1123-4f39-adb6-548f5d019936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:47.951120 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b353b6d-1123-4f39-adb6-548f5d019936 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:47.951406 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b353b6d-1123-4f39-adb6-548f5d019936 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:47.951683 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b353b6d-1123-4f39-adb6-548f5d019936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:47.953201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f2619-057c-42ef-a54e-f91ec05e3800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:47.955483 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f2619-057c-42ef-a54e-f91ec05e3800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:47.956289 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f2619-057c-42ef-a54e-f91ec05e3800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:47.956420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f2619-057c-42ef-a54e-f91ec05e3800 admin admin] PATH_INFO: `/v3/projects/d225e79dd1164e5f92160737fdf84c31` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:47.960979 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f2619-057c-42ef-a54e-f91ec05e3800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d225e79dd1164e5f92160737fdf84c31)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:47.967791 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-2e5f2619-057c-42ef-a54e-f91ec05e3800 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 18:04:47.967791 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: warnings.warn(msg) May 26 18:04:47.967791 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: May 26 18:04:47.971544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b353b6d-1123-4f39-adb6-548f5d019936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:47.972133 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f2619-057c-42ef-a54e-f91ec05e3800 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:47.976091 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:47.977457 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b353b6d-1123-4f39-adb6-548f5d019936 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:47.994818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:48.019846 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 814/1651] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:04:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:48.288936 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2e5f2619-057c-42ef-a54e-f91ec05e3800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd225e79dd1164e5f92160737fdf84c31', 'request_id': 'req-2e5f2619-057c-42ef-a54e-f91ec05e3800'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:48.307306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2e5f2619-057c-42ef-a54e-f91ec05e3800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd225e79dd1164e5f92160737fdf84c31', 'request_id': 'req-2e5f2619-057c-42ef-a54e-f91ec05e3800'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:48.332512 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2e5f2619-057c-42ef-a54e-f91ec05e3800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd225e79dd1164e5f92160737fdf84c31', 'request_id': 'req-2e5f2619-057c-42ef-a54e-f91ec05e3800'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:48.359375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2e5f2619-057c-42ef-a54e-f91ec05e3800 admin admin] The token cache is being invalidate because project d225e79dd1164e5f92160737fdf84c31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:04:48.359677 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2e5f2619-057c-42ef-a54e-f91ec05e3800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:48.362411 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2e5f2619-057c-42ef-a54e-f91ec05e3800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:48.369130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 838/1652] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:04:47 2022] DELETE /identity/v3/projects/d225e79dd1164e5f92160737fdf84c31 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:48.370388 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108c3e4-7d2a-4408-938b-82a79c51aee8 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:48.370879 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108c3e4-7d2a-4408-938b-82a79c51aee8 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:48.370949 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108c3e4-7d2a-4408-938b-82a79c51aee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:48.411492 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-e108c3e4-7d2a-4408-938b-82a79c51aee8 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:48.412753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-e108c3e4-7d2a-4408-938b-82a79c51aee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:48.537586 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 815/1653] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:04:48.547424 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ff7e1-55d0-4ae2-b2c7-0ab2a78bfd2d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:48.548421 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ff7e1-55d0-4ae2-b2c7-0ab2a78bfd2d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:48.548954 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ff7e1-55d0-4ae2-b2c7-0ab2a78bfd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:48.590848 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f96ff7e1-55d0-4ae2-b2c7-0ab2a78bfd2d None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:48.592583 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f96ff7e1-55d0-4ae2-b2c7-0ab2a78bfd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:48.613115 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 839/1654] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:48.617721 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108c3e4-7d2a-4408-938b-82a79c51aee8 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:48.633292 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108c3e4-7d2a-4408-938b-82a79c51aee8 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:48.639807 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a500c1-908a-4aef-84a9-94109013ae1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:48.640783 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a500c1-908a-4aef-84a9-94109013ae1d admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:48.641394 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a500c1-908a-4aef-84a9-94109013ae1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:48.641994 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a500c1-908a-4aef-84a9-94109013ae1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:48.643353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a500c1-908a-4aef-84a9-94109013ae1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:48.643815 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a500c1-908a-4aef-84a9-94109013ae1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:48.645785 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a500c1-908a-4aef-84a9-94109013ae1d admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:48.663290 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 816/1655] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:48.668568 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ff7e1-55d0-4ae2-b2c7-0ab2a78bfd2d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:48.681793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ff7e1-55d0-4ae2-b2c7-0ab2a78bfd2d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:48.687450 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a389e9-95df-49c1-98ff-e9c245d455ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:48.689002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a389e9-95df-49c1-98ff-e9c245d455ed admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:48.689894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a389e9-95df-49c1-98ff-e9c245d455ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:48.690362 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a389e9-95df-49c1-98ff-e9c245d455ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:48.691776 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a389e9-95df-49c1-98ff-e9c245d455ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:48.693985 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a389e9-95df-49c1-98ff-e9c245d455ed admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:48.743940 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 840/1656] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:04:48 2022] POST /identity/v3/projects => generated 405 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:04:48.748170 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a500c1-908a-4aef-84a9-94109013ae1d admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:48.779982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a500c1-908a-4aef-84a9-94109013ae1d admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:48.944771 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5bb65-f65d-4103-9c9f-55092de53d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:48.948995 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5bb65-f65d-4103-9c9f-55092de53d2d admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:48.949532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5bb65-f65d-4103-9c9f-55092de53d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:48.950589 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5bb65-f65d-4103-9c9f-55092de53d2d admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:48.952377 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c5bb65-f65d-4103-9c9f-55092de53d2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:48.959415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c5bb65-f65d-4103-9c9f-55092de53d2d admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:49.013684 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 817/1657] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:04:48 2022] POST /identity/v3/users => generated 303 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:49.017619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a389e9-95df-49c1-98ff-e9c245d455ed admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:49.031533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a389e9-95df-49c1-98ff-e9c245d455ed admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:49.041095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64eb4a-7471-4cad-8af3-a0e4aff36eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:49.041964 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64eb4a-7471-4cad-8af3-a0e4aff36eac admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:49.042393 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64eb4a-7471-4cad-8af3-a0e4aff36eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:49.042681 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64eb4a-7471-4cad-8af3-a0e4aff36eac admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:49.043555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64eb4a-7471-4cad-8af3-a0e4aff36eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:49.045388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64eb4a-7471-4cad-8af3-a0e4aff36eac admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:49.073527 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 841/1658] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:49.077543 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5bb65-f65d-4103-9c9f-55092de53d2d admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:49.100697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5bb65-f65d-4103-9c9f-55092de53d2d admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:49.125810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705c432-3005-4dd5-af31-8e49a76b01d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:49.126956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705c432-3005-4dd5-af31-8e49a76b01d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:49.128104 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705c432-3005-4dd5-af31-8e49a76b01d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:49.128623 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705c432-3005-4dd5-af31-8e49a76b01d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:49.129641 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705c432-3005-4dd5-af31-8e49a76b01d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:49.134705 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705c432-3005-4dd5-af31-8e49a76b01d4 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:49.153722 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 818/1659] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:49.159218 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64eb4a-7471-4cad-8af3-a0e4aff36eac admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:49.175889 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64eb4a-7471-4cad-8af3-a0e4aff36eac admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:49.193104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e1524-5b05-4770-8cd2-151fe9ae8833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:49.194842 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e1524-5b05-4770-8cd2-151fe9ae8833 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:49.197309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e1524-5b05-4770-8cd2-151fe9ae8833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:49.197855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e1524-5b05-4770-8cd2-151fe9ae8833 admin admin] PATH_INFO: `/v3/projects/642d4ba1d79e47fe83ac853019fc9539/users/6fdb0a2dac5d4daf8a180d7c88db5332/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:49.228892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e1524-5b05-4770-8cd2-151fe9ae8833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=642d4ba1d79e47fe83ac853019fc9539, user_id=6fdb0a2dac5d4daf8a180d7c88db5332, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:49.236994 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e1524-5b05-4770-8cd2-151fe9ae8833 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:49.293024 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 842/1660] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:04:49 2022] PUT /identity/v3/projects/642d4ba1d79e47fe83ac853019fc9539/users/6fdb0a2dac5d4daf8a180d7c88db5332/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:49.308586 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e82d0-63ae-4d12-9f4d-c93497edb042 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:49.308886 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e82d0-63ae-4d12-9f4d-c93497edb042 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:49.309213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e82d0-63ae-4d12-9f4d-c93497edb042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:49.348805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-167e82d0-63ae-4d12-9f4d-c93497edb042 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:49.357877 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-167e82d0-63ae-4d12-9f4d-c93497edb042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:49.545166 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 819/1661] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:49.558247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e1524-5b05-4770-8cd2-151fe9ae8833 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:49.560029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-088e1524-5b05-4770-8cd2-151fe9ae8833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:49.718865 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e1524-5b05-4770-8cd2-151fe9ae8833 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:49.751300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb260a21-fac1-4a83-aa5b-1c1f3a50454f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:49.752366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb260a21-fac1-4a83-aa5b-1c1f3a50454f service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:49.752669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb260a21-fac1-4a83-aa5b-1c1f3a50454f service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:49.752954 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb260a21-fac1-4a83-aa5b-1c1f3a50454f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:49.769182 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb260a21-fac1-4a83-aa5b-1c1f3a50454f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:49.770968 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb260a21-fac1-4a83-aa5b-1c1f3a50454f service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:49.799511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 843/1662] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:04:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:49.899218 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e82d0-63ae-4d12-9f4d-c93497edb042 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:49.903627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-167e82d0-63ae-4d12-9f4d-c93497edb042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:49.939360 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:49.975228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e82d0-63ae-4d12-9f4d-c93497edb042 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:50.000609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a4ad4-4170-4682-bbb3-0a545319cb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:50.005339 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a4ad4-4170-4682-bbb3-0a545319cb0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:50.005847 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a4ad4-4170-4682-bbb3-0a545319cb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:50.006304 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a4ad4-4170-4682-bbb3-0a545319cb0a admin admin] PATH_INFO: `/v3/projects/59bf92ddd0db48cb81c512a9a2b5b6cf` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:50.024694 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a4ad4-4170-4682-bbb3-0a545319cb0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=59bf92ddd0db48cb81c512a9a2b5b6cf)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:50.026973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a4ad4-4170-4682-bbb3-0a545319cb0a admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:50.097390 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb260a21-fac1-4a83-aa5b-1c1f3a50454f service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:50.099082 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fb260a21-fac1-4a83-aa5b-1c1f3a50454f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:50.132980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb260a21-fac1-4a83-aa5b-1c1f3a50454f service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:50.187524 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3346911-9dd2-40b5-ae6f-aa573f78659b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:50.190318 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3346911-9dd2-40b5-ae6f-aa573f78659b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:50.191644 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3346911-9dd2-40b5-ae6f-aa573f78659b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:50.199376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3346911-9dd2-40b5-ae6f-aa573f78659b admin admin] PATH_INFO: `/v3/users/6d54cc1979ca43b7818bd0d21c1d735f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:50.202605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3346911-9dd2-40b5-ae6f-aa573f78659b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d54cc1979ca43b7818bd0d21c1d735f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:50.214642 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3346911-9dd2-40b5-ae6f-aa573f78659b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:50.270413 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-921a4ad4-4170-4682-bbb3-0a545319cb0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59bf92ddd0db48cb81c512a9a2b5b6cf', 'request_id': 'req-921a4ad4-4170-4682-bbb3-0a545319cb0a'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:50.295122 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:50.301022 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-921a4ad4-4170-4682-bbb3-0a545319cb0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59bf92ddd0db48cb81c512a9a2b5b6cf', 'request_id': 'req-921a4ad4-4170-4682-bbb3-0a545319cb0a'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:50.334466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-921a4ad4-4170-4682-bbb3-0a545319cb0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59bf92ddd0db48cb81c512a9a2b5b6cf', 'request_id': 'req-921a4ad4-4170-4682-bbb3-0a545319cb0a'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:50.377867 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b3346911-9dd2-40b5-ae6f-aa573f78659b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d54cc1979ca43b7818bd0d21c1d735f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:50.403129 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b3346911-9dd2-40b5-ae6f-aa573f78659b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d54cc1979ca43b7818bd0d21c1d735f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:50.423768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-921a4ad4-4170-4682-bbb3-0a545319cb0a admin admin] The token cache is being invalidate because project 59bf92ddd0db48cb81c512a9a2b5b6cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:04:50.424467 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-921a4ad4-4170-4682-bbb3-0a545319cb0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:50.426409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-921a4ad4-4170-4682-bbb3-0a545319cb0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:50.428800 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 820/1663] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:04:49 2022] DELETE /identity/v3/projects/59bf92ddd0db48cb81c512a9a2b5b6cf => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:50.432831 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a4ad4-4170-4682-bbb3-0a545319cb0a admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:50.437986 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-921a4ad4-4170-4682-bbb3-0a545319cb0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:50.441984 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b3346911-9dd2-40b5-ae6f-aa573f78659b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d54cc1979ca43b7818bd0d21c1d735f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:50.446934 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b3346911-9dd2-40b5-ae6f-aa573f78659b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d54cc1979ca43b7818bd0d21c1d735f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:50.449128 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b3346911-9dd2-40b5-ae6f-aa573f78659b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d54cc1979ca43b7818bd0d21c1d735f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:50.481463 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 844/1664] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:04:50 2022] DELETE /identity/v3/users/6d54cc1979ca43b7818bd0d21c1d735f => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:50.487596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bbfd0-036b-4aa6-8bb4-096ecf46985d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:50.488108 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bbfd0-036b-4aa6-8bb4-096ecf46985d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:50.488530 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bbfd0-036b-4aa6-8bb4-096ecf46985d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:50.538754 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d75bbfd0-036b-4aa6-8bb4-096ecf46985d None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:50.542400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d75bbfd0-036b-4aa6-8bb4-096ecf46985d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:50.672567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a4ad4-4170-4682-bbb3-0a545319cb0a admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:50.750859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 845/1665] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:50.822584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75bbfd0-036b-4aa6-8bb4-096ecf46985d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:50.824750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d75bbfd0-036b-4aa6-8bb4-096ecf46985d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:50.875155 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d299a-29fd-4f4b-88d3-b3a5efc8f17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:50.876931 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d299a-29fd-4f4b-88d3-b3a5efc8f17f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:50.878624 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d299a-29fd-4f4b-88d3-b3a5efc8f17f admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:50.878936 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d299a-29fd-4f4b-88d3-b3a5efc8f17f admin admin] PATH_INFO: `/v3/users/40b87a7069984a5bac4215179d3e33ae` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:50.882513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d299a-29fd-4f4b-88d3-b3a5efc8f17f admin admin] RBAC: Authorizing `identity:delete_user(user_id=40b87a7069984a5bac4215179d3e33ae)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:50.883624 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: WARNING py.warnings [None req-d44d299a-29fd-4f4b-88d3-b3a5efc8f17f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 18:04:50.883624 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: warnings.warn(msg) May 26 18:04:50.883624 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: May 26 18:04:50.890155 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d299a-29fd-4f4b-88d3-b3a5efc8f17f admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:51.007396 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75bbfd0-036b-4aa6-8bb4-096ecf46985d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:51.022800 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db287c-086f-4018-b56d-8614daed27b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:51.024577 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db287c-086f-4018-b56d-8614daed27b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:51.025822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db287c-086f-4018-b56d-8614daed27b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:51.026095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db287c-086f-4018-b56d-8614daed27b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:51.059124 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:51.089027 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db287c-086f-4018-b56d-8614daed27b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:51.090581 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db287c-086f-4018-b56d-8614daed27b4 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:51.109988 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d44d299a-29fd-4f4b-88d3-b3a5efc8f17f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40b87a7069984a5bac4215179d3e33ae'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:51.146249 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d44d299a-29fd-4f4b-88d3-b3a5efc8f17f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40b87a7069984a5bac4215179d3e33ae'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:51.148894 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d44d299a-29fd-4f4b-88d3-b3a5efc8f17f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40b87a7069984a5bac4215179d3e33ae'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:51.149791 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d44d299a-29fd-4f4b-88d3-b3a5efc8f17f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40b87a7069984a5bac4215179d3e33ae'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:51.150613 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 846/1666] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:04:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:51.194211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d44d299a-29fd-4f4b-88d3-b3a5efc8f17f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40b87a7069984a5bac4215179d3e33ae'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:51.299691 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 821/1667] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:04:50 2022] DELETE /identity/v3/users/40b87a7069984a5bac4215179d3e33ae => generated 0 bytes in 867 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:52.036823 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2f5d1-f48e-4fb8-9dd9-3218178d1435 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:52.037379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2f5d1-f48e-4fb8-9dd9-3218178d1435 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:52.037456 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2f5d1-f48e-4fb8-9dd9-3218178d1435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:52.093489 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3ab2f5d1-f48e-4fb8-9dd9-3218178d1435 None None] MFA Rules not processed for user `3e06d1b0e1a9472db764f4be7e42fdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:52.097780 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3ab2f5d1-f48e-4fb8-9dd9-3218178d1435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:52.239246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:52.287196 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 847/1668] 10.222.0.76 () {58 vars in 1009 bytes} [Thu May 26 18:04:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:52.623276 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d299a-29fd-4f4b-88d3-b3a5efc8f17f admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:52.624521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-d44d299a-29fd-4f4b-88d3-b3a5efc8f17f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:52.837441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d299a-29fd-4f4b-88d3-b3a5efc8f17f admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:52.863314 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4828ea-4954-4b34-9a59-9de67e2041e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ca774e864ca44cfb187e38bc8a8f6ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:52.868399 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4828ea-4954-4b34-9a59-9de67e2041e1 service swift] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:52.868802 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4828ea-4954-4b34-9a59-9de67e2041e1 service swift] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:52.869090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4828ea-4954-4b34-9a59-9de67e2041e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:52.888133 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b24a4-90bc-435e-a96e-1a5f6be561ad None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:52.888460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b24a4-90bc-435e-a96e-1a5f6be561ad None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:52.888691 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b24a4-90bc-435e-a96e-1a5f6be561ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:52.927276 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4828ea-4954-4b34-9a59-9de67e2041e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:52.934343 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4828ea-4954-4b34-9a59-9de67e2041e1 service swift] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:52.979768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 822/1669] 10.222.0.76 () {60 vars in 1384 bytes} [Thu May 26 18:04:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 18:04:52.993028 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4828ea-4954-4b34-9a59-9de67e2041e1 service swift] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:53.000120 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-6b4828ea-4954-4b34-9a59-9de67e2041e1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:53.005061 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-940b24a4-90bc-435e-a96e-1a5f6be561ad None None] MFA Rules not processed for user `6fdb0a2dac5d4daf8a180d7c88db5332`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:53.017445 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-940b24a4-90bc-435e-a96e-1a5f6be561ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:53.199409 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 848/1670] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:52 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:53.200335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b24a4-90bc-435e-a96e-1a5f6be561ad None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:53.206559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-940b24a4-90bc-435e-a96e-1a5f6be561ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:53.321136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4828ea-4954-4b34-9a59-9de67e2041e1 service swift] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:53.342819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda34c51-365d-4b2b-b7ce-16cd2cea52a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:53.346004 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda34c51-365d-4b2b-b7ce-16cd2cea52a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:53.348158 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda34c51-365d-4b2b-b7ce-16cd2cea52a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:53.348393 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda34c51-365d-4b2b-b7ce-16cd2cea52a9 admin admin] PATH_INFO: `/v3/users/5389ef3de3094a6ba85f928312c99dde` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:53.354966 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda34c51-365d-4b2b-b7ce-16cd2cea52a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5389ef3de3094a6ba85f928312c99dde)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:53.361774 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda34c51-365d-4b2b-b7ce-16cd2cea52a9 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:53.367141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b24a4-90bc-435e-a96e-1a5f6be561ad None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:53.379612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0198579-cfc2-4a25-823c-6d5528c2fe69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:53.380315 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0198579-cfc2-4a25-823c-6d5528c2fe69 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:53.380646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0198579-cfc2-4a25-823c-6d5528c2fe69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:53.380931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0198579-cfc2-4a25-823c-6d5528c2fe69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:53.381815 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0198579-cfc2-4a25-823c-6d5528c2fe69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:53.383932 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0198579-cfc2-4a25-823c-6d5528c2fe69 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:53.464603 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 849/1671] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:04:53 2022] POST /identity/v3/projects => generated 403 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:53.469765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0198579-cfc2-4a25-823c-6d5528c2fe69 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:53.517371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:53.523223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-bda34c51-365d-4b2b-b7ce-16cd2cea52a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5389ef3de3094a6ba85f928312c99dde'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:53.550970 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-bda34c51-365d-4b2b-b7ce-16cd2cea52a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5389ef3de3094a6ba85f928312c99dde'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:53.552543 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-bda34c51-365d-4b2b-b7ce-16cd2cea52a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5389ef3de3094a6ba85f928312c99dde'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:53.554762 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-bda34c51-365d-4b2b-b7ce-16cd2cea52a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5389ef3de3094a6ba85f928312c99dde'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:53.584049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-bda34c51-365d-4b2b-b7ce-16cd2cea52a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5389ef3de3094a6ba85f928312c99dde'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:53.665322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 823/1672] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:04:52 2022] DELETE /identity/v3/users/5389ef3de3094a6ba85f928312c99dde => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:53.665322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda34c51-365d-4b2b-b7ce-16cd2cea52a9 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:53.669164 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-bda34c51-365d-4b2b-b7ce-16cd2cea52a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:53.704793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0198579-cfc2-4a25-823c-6d5528c2fe69 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:53.733946 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:53.891217 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda34c51-365d-4b2b-b7ce-16cd2cea52a9 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:53.920534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970f2aa-17ba-4ab3-a0d0-54dba0dda4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:53.921408 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970f2aa-17ba-4ab3-a0d0-54dba0dda4c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:53.921520 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970f2aa-17ba-4ab3-a0d0-54dba0dda4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:53.921746 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970f2aa-17ba-4ab3-a0d0-54dba0dda4c7 admin admin] PATH_INFO: `/v3/projects/4d4d85df62fe46a9af5f2f5e4b7ab971` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:53.930428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664451a-65d1-48af-b883-b38a1ea8c335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:53.933069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664451a-65d1-48af-b883-b38a1ea8c335 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:53.933364 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664451a-65d1-48af-b883-b38a1ea8c335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:53.933651 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664451a-65d1-48af-b883-b38a1ea8c335 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:53.934404 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664451a-65d1-48af-b883-b38a1ea8c335 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:53.940236 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970f2aa-17ba-4ab3-a0d0-54dba0dda4c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d4d85df62fe46a9af5f2f5e4b7ab971)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:53.943527 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970f2aa-17ba-4ab3-a0d0-54dba0dda4c7 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:53.949288 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664451a-65d1-48af-b883-b38a1ea8c335 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:54.029396 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 850/1673] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:04:53 2022] POST /identity/v3/users => generated 308 bytes in 560 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:54.031752 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775eedd6-4704-4321-b786-13786812157d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:54.038892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775eedd6-4704-4321-b786-13786812157d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:54.039482 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775eedd6-4704-4321-b786-13786812157d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:54.105164 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-775eedd6-4704-4321-b786-13786812157d None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:54.105164 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-775eedd6-4704-4321-b786-13786812157d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:54.122473 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 851/1674] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:54.126041 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775eedd6-4704-4321-b786-13786812157d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:54.133720 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-775eedd6-4704-4321-b786-13786812157d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:54.178453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775eedd6-4704-4321-b786-13786812157d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:54.204301 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-c970f2aa-17ba-4ab3-a0d0-54dba0dda4c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d4d85df62fe46a9af5f2f5e4b7ab971', 'request_id': 'req-c970f2aa-17ba-4ab3-a0d0-54dba0dda4c7'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:54.227521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-c970f2aa-17ba-4ab3-a0d0-54dba0dda4c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d4d85df62fe46a9af5f2f5e4b7ab971', 'request_id': 'req-c970f2aa-17ba-4ab3-a0d0-54dba0dda4c7'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:54.240565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-c970f2aa-17ba-4ab3-a0d0-54dba0dda4c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d4d85df62fe46a9af5f2f5e4b7ab971', 'request_id': 'req-c970f2aa-17ba-4ab3-a0d0-54dba0dda4c7'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:54.280484 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-c970f2aa-17ba-4ab3-a0d0-54dba0dda4c7 admin admin] The token cache is being invalidate because project 4d4d85df62fe46a9af5f2f5e4b7ab971 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:04:54.280841 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-c970f2aa-17ba-4ab3-a0d0-54dba0dda4c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:54.282829 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-c970f2aa-17ba-4ab3-a0d0-54dba0dda4c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:54.287371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 824/1675] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:04:53 2022] DELETE /identity/v3/projects/4d4d85df62fe46a9af5f2f5e4b7ab971 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:54.293688 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36801f3c-d3a1-4eb4-bcb9-972f6c75b927 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:54.294345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36801f3c-d3a1-4eb4-bcb9-972f6c75b927 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:54.296788 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36801f3c-d3a1-4eb4-bcb9-972f6c75b927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:54.360071 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-36801f3c-d3a1-4eb4-bcb9-972f6c75b927 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:54.369184 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-36801f3c-d3a1-4eb4-bcb9-972f6c75b927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:54.388289 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03beb68-fd3c-4e00-ae62-961ee0e0d71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:54.391777 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03beb68-fd3c-4e00-ae62-961ee0e0d71c admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:54.393055 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03beb68-fd3c-4e00-ae62-961ee0e0d71c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:54.394275 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03beb68-fd3c-4e00-ae62-961ee0e0d71c admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:54.397522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03beb68-fd3c-4e00-ae62-961ee0e0d71c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:54.404580 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03beb68-fd3c-4e00-ae62-961ee0e0d71c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:54.433416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 852/1676] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:54.436550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03beb68-fd3c-4e00-ae62-961ee0e0d71c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:54.442380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a03beb68-fd3c-4e00-ae62-961ee0e0d71c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:54.673525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03beb68-fd3c-4e00-ae62-961ee0e0d71c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:54.684854 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ae263-5a2c-47bd-9f5e-c1dfd9463926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:54.685662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ae263-5a2c-47bd-9f5e-c1dfd9463926 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:54.686042 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ae263-5a2c-47bd-9f5e-c1dfd9463926 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:54.686536 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ae263-5a2c-47bd-9f5e-c1dfd9463926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:54.694690 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-de7ae263-5a2c-47bd-9f5e-c1dfd9463926 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:54.726260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ae263-5a2c-47bd-9f5e-c1dfd9463926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:54.728079 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ae263-5a2c-47bd-9f5e-c1dfd9463926 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:54.768307 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 853/1677] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:04:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:54.769305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ae263-5a2c-47bd-9f5e-c1dfd9463926 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:54.771222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-de7ae263-5a2c-47bd-9f5e-c1dfd9463926 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:54.811852 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ae263-5a2c-47bd-9f5e-c1dfd9463926 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:54.833105 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35390c1-3522-4919-94ce-ff0a0d0cd160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:54.834152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35390c1-3522-4919-94ce-ff0a0d0cd160 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:54.834525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35390c1-3522-4919-94ce-ff0a0d0cd160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:54.834826 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35390c1-3522-4919-94ce-ff0a0d0cd160 admin admin] PATH_INFO: `/v3/projects/3758efd2fdea4a34b0fae1496bb7c14a/users/e5e66ed4a9b346a68bb70206027946a9/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:54.899131 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35390c1-3522-4919-94ce-ff0a0d0cd160 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3758efd2fdea4a34b0fae1496bb7c14a, user_id=e5e66ed4a9b346a68bb70206027946a9, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:54.903051 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35390c1-3522-4919-94ce-ff0a0d0cd160 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:54.941312 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 854/1678] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:04:54 2022] PUT /identity/v3/projects/3758efd2fdea4a34b0fae1496bb7c14a/users/e5e66ed4a9b346a68bb70206027946a9/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:54.942878 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35390c1-3522-4919-94ce-ff0a0d0cd160 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:54.949821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b35390c1-3522-4919-94ce-ff0a0d0cd160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:54.980212 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:55.068496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:55.128682 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 825/1679] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 837 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:55.137848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36801f3c-d3a1-4eb4-bcb9-972f6c75b927 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:55.175342 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36801f3c-d3a1-4eb4-bcb9-972f6c75b927 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:55.191983 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c92a73-31df-4b10-9997-76309741f052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:55.192764 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c92a73-31df-4b10-9997-76309741f052 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:55.192979 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c92a73-31df-4b10-9997-76309741f052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:55.193260 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c92a73-31df-4b10-9997-76309741f052 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:55.194012 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c92a73-31df-4b10-9997-76309741f052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:55.195750 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c92a73-31df-4b10-9997-76309741f052 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:55.208502 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35390c1-3522-4919-94ce-ff0a0d0cd160 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:55.230558 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99409cb-4166-4c19-8d12-c0a89137925c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:55.231454 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99409cb-4166-4c19-8d12-c0a89137925c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:55.234862 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99409cb-4166-4c19-8d12-c0a89137925c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:55.235826 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99409cb-4166-4c19-8d12-c0a89137925c admin admin] PATH_INFO: `/v3/projects/1ff84f5142c24aa4a50fbcb6510a5624` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:55.245114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99409cb-4166-4c19-8d12-c0a89137925c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ff84f5142c24aa4a50fbcb6510a5624)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:55.247176 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99409cb-4166-4c19-8d12-c0a89137925c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:55.249984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 826/1680] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:55.255829 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23e7cf-d512-4914-a8d4-2644f3a6f0ac None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:55.257357 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23e7cf-d512-4914-a8d4-2644f3a6f0ac None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:55.257655 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23e7cf-d512-4914-a8d4-2644f3a6f0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:55.331599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-ba23e7cf-d512-4914-a8d4-2644f3a6f0ac None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:55.336621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-ba23e7cf-d512-4914-a8d4-2644f3a6f0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:55.386410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 827/1681] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:55.390837 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23e7cf-d512-4914-a8d4-2644f3a6f0ac None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:55.442675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23e7cf-d512-4914-a8d4-2644f3a6f0ac None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:55.504340 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b99409cb-4166-4c19-8d12-c0a89137925c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ff84f5142c24aa4a50fbcb6510a5624', 'request_id': 'req-b99409cb-4166-4c19-8d12-c0a89137925c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:55.529495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b99409cb-4166-4c19-8d12-c0a89137925c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ff84f5142c24aa4a50fbcb6510a5624', 'request_id': 'req-b99409cb-4166-4c19-8d12-c0a89137925c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:55.549572 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b99409cb-4166-4c19-8d12-c0a89137925c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ff84f5142c24aa4a50fbcb6510a5624', 'request_id': 'req-b99409cb-4166-4c19-8d12-c0a89137925c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:55.598451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b99409cb-4166-4c19-8d12-c0a89137925c admin admin] The token cache is being invalidate because project 1ff84f5142c24aa4a50fbcb6510a5624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:04:55.598451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b99409cb-4166-4c19-8d12-c0a89137925c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:55.600013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b99409cb-4166-4c19-8d12-c0a89137925c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:55.602308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 855/1682] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:04:54 2022] DELETE /identity/v3/projects/1ff84f5142c24aa4a50fbcb6510a5624 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:55.603198 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99409cb-4166-4c19-8d12-c0a89137925c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:55.611814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b99409cb-4166-4c19-8d12-c0a89137925c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:55.774751 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9c2ea-9def-4259-aa70-edac6491f8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:55.777501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9c2ea-9def-4259-aa70-edac6491f8b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:55.780449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9c2ea-9def-4259-aa70-edac6491f8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:55.787714 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9c2ea-9def-4259-aa70-edac6491f8b0 admin admin] PATH_INFO: `/v3/projects/3758efd2fdea4a34b0fae1496bb7c14a/users/e5e66ed4a9b346a68bb70206027946a9/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:55.797584 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9c2ea-9def-4259-aa70-edac6491f8b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3758efd2fdea4a34b0fae1496bb7c14a, user_id=e5e66ed4a9b346a68bb70206027946a9, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:55.805873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99409cb-4166-4c19-8d12-c0a89137925c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:55.806316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9c2ea-9def-4259-aa70-edac6491f8b0 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:55.821604 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aeeceb-f60b-4c16-be0c-821f0e5cd1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:55.825282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aeeceb-f60b-4c16-be0c-821f0e5cd1f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:55.825762 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aeeceb-f60b-4c16-be0c-821f0e5cd1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:55.826176 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aeeceb-f60b-4c16-be0c-821f0e5cd1f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:55.827090 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aeeceb-f60b-4c16-be0c-821f0e5cd1f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:55.828633 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aeeceb-f60b-4c16-be0c-821f0e5cd1f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:55.834708 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 828/1683] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:04:55 2022] PUT /identity/v3/projects/3758efd2fdea4a34b0fae1496bb7c14a/users/e5e66ed4a9b346a68bb70206027946a9/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:55.835110 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aeeceb-f60b-4c16-be0c-821f0e5cd1f4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:55.837201 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b89bf-389b-4207-bb9c-3b8c61fde628 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:55.840209 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b89bf-389b-4207-bb9c-3b8c61fde628 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:55.841043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b89bf-389b-4207-bb9c-3b8c61fde628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:55.871920 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 856/1684] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:55.875721 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba212d42-d7b4-4ae1-9994-a870e88bd966 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:55.876351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba212d42-d7b4-4ae1-9994-a870e88bd966 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:55.876917 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba212d42-d7b4-4ae1-9994-a870e88bd966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:55.892693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-2b9b89bf-389b-4207-bb9c-3b8c61fde628 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:55.897285 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-2b9b89bf-389b-4207-bb9c-3b8c61fde628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:55.960123 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ba212d42-d7b4-4ae1-9994-a870e88bd966 None None] MFA Rules not processed for user `f6b5113ea0f4484ba91a5f0e6184ca35`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:55.964003 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ba212d42-d7b4-4ae1-9994-a870e88bd966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:56.031417 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:56.085922 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 829/1685] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:56.088052 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b89bf-389b-4207-bb9c-3b8c61fde628 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:56.103890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b89bf-389b-4207-bb9c-3b8c61fde628 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:56.110981 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d98aa-cdbf-4a29-a274-b5698612e50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:56.119708 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d98aa-cdbf-4a29-a274-b5698612e50c admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:56.120391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d98aa-cdbf-4a29-a274-b5698612e50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:56.120903 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d98aa-cdbf-4a29-a274-b5698612e50c admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:56.121844 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d98aa-cdbf-4a29-a274-b5698612e50c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:56.124401 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d98aa-cdbf-4a29-a274-b5698612e50c admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:56.155095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 857/1686] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:56.155095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f51e5c-62a7-44c4-b550-a6435661b8b7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:56.155095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f51e5c-62a7-44c4-b550-a6435661b8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:56.155095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f51e5c-62a7-44c4-b550-a6435661b8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:56.192931 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 830/1687] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:04:56 2022] POST /identity/v3/projects => generated 369 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:56.194397 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d98aa-cdbf-4a29-a274-b5698612e50c admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:56.198100 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-582d98aa-cdbf-4a29-a274-b5698612e50c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:56.223587 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-69f51e5c-62a7-44c4-b550-a6435661b8b7 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:56.230070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-69f51e5c-62a7-44c4-b550-a6435661b8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:56.353792 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:56.409463 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d98aa-cdbf-4a29-a274-b5698612e50c admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:56.419033 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d80b2-8238-4090-8bbb-11a03d670bbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:56.420281 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d80b2-8238-4090-8bbb-11a03d670bbb service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:56.421343 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d80b2-8238-4090-8bbb-11a03d670bbb service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:56.421990 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d80b2-8238-4090-8bbb-11a03d670bbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:56.500375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 858/1688] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:56.503047 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f51e5c-62a7-44c4-b550-a6435661b8b7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:56.541448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f51e5c-62a7-44c4-b550-a6435661b8b7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:56.561302 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f596dc-b192-4344-8cd5-9a50e92187b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:56.561866 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f596dc-b192-4344-8cd5-9a50e92187b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:56.565156 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f596dc-b192-4344-8cd5-9a50e92187b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:56.565585 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f596dc-b192-4344-8cd5-9a50e92187b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:56.568116 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f596dc-b192-4344-8cd5-9a50e92187b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:56.581167 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f596dc-b192-4344-8cd5-9a50e92187b4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:56.628935 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d80b2-8238-4090-8bbb-11a03d670bbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:56.637222 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d80b2-8238-4090-8bbb-11a03d670bbb service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:56.668032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 859/1689] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:04:56 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:56.671161 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f596dc-b192-4344-8cd5-9a50e92187b4 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:56.681329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 831/1690] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:04:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:56.683166 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d80b2-8238-4090-8bbb-11a03d670bbb service nova] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:56.707073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f596dc-b192-4344-8cd5-9a50e92187b4 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:56.712853 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d80b2-8238-4090-8bbb-11a03d670bbb service nova] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:56.720769 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9b57b-e547-4bf6-80ed-3faf67af5f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:56.721703 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9b57b-e547-4bf6-80ed-3faf67af5f7d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:56.723008 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9b57b-e547-4bf6-80ed-3faf67af5f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:56.723869 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9b57b-e547-4bf6-80ed-3faf67af5f7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:56.725860 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9b57b-e547-4bf6-80ed-3faf67af5f7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:56.727531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b64df-c368-4bdd-9c6a-26ec6ca42b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:56.728666 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b64df-c368-4bdd-9c6a-26ec6ca42b39 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:56.729312 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9b57b-e547-4bf6-80ed-3faf67af5f7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:56.733190 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b64df-c368-4bdd-9c6a-26ec6ca42b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:56.733190 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b64df-c368-4bdd-9c6a-26ec6ca42b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:56.733190 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b64df-c368-4bdd-9c6a-26ec6ca42b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:56.734755 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9b57b-e547-4bf6-80ed-3faf67af5f7d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:56.738828 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b64df-c368-4bdd-9c6a-26ec6ca42b39 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:56.769427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 832/1691] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:56.778150 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b64df-c368-4bdd-9c6a-26ec6ca42b39 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:56.780563 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 860/1692] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:56.818229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b64df-c368-4bdd-9c6a-26ec6ca42b39 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:56.818634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9b57b-e547-4bf6-80ed-3faf67af5f7d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:56.826985 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece325d-4dd4-4fd4-8c55-5b96644e1e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:56.827868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece325d-4dd4-4fd4-8c55-5b96644e1e49 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:56.828290 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece325d-4dd4-4fd4-8c55-5b96644e1e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:56.829235 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece325d-4dd4-4fd4-8c55-5b96644e1e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:56.830504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece325d-4dd4-4fd4-8c55-5b96644e1e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:56.835774 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece325d-4dd4-4fd4-8c55-5b96644e1e49 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:56.846523 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9b57b-e547-4bf6-80ed-3faf67af5f7d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:56.852678 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a1712-ee1a-4095-bd1e-f3c8886c2f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:56.853368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a1712-ee1a-4095-bd1e-f3c8886c2f1f admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:56.853659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a1712-ee1a-4095-bd1e-f3c8886c2f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:56.853940 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a1712-ee1a-4095-bd1e-f3c8886c2f1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:56.854848 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a1712-ee1a-4095-bd1e-f3c8886c2f1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:56.874058 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a1712-ee1a-4095-bd1e-f3c8886c2f1f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:56.879093 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 833/1693] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:56.890037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece325d-4dd4-4fd4-8c55-5b96644e1e49 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:56.926292 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece325d-4dd4-4fd4-8c55-5b96644e1e49 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:56.948903 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672ff0d-c694-4563-9830-9a98c6da3c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:56.949934 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672ff0d-c694-4563-9830-9a98c6da3c35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:56.950519 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 861/1694] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:04:56 2022] POST /identity/v3/projects => generated 365 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:56.952544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672ff0d-c694-4563-9830-9a98c6da3c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:56.952951 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672ff0d-c694-4563-9830-9a98c6da3c35 admin admin] PATH_INFO: `/v3/projects/3a2d669b5cd141408bb971741e17dd53/users/38aeb7043f29468fb4f4429376bf3d7b/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:56.954121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a1712-ee1a-4095-bd1e-f3c8886c2f1f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:57.006939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672ff0d-c694-4563-9830-9a98c6da3c35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a2d669b5cd141408bb971741e17dd53, user_id=38aeb7043f29468fb4f4429376bf3d7b, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:57.011784 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672ff0d-c694-4563-9830-9a98c6da3c35 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:57.075891 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 834/1695] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:04:56 2022] PUT /identity/v3/projects/3a2d669b5cd141408bb971741e17dd53/users/38aeb7043f29468fb4f4429376bf3d7b/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:57.080257 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a69ed0-c559-4502-a190-efe15b398443 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:57.080653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a69ed0-c559-4502-a190-efe15b398443 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:57.081048 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a69ed0-c559-4502-a190-efe15b398443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:57.190504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-11a69ed0-c559-4502-a190-efe15b398443 None None] MFA Rules not processed for user `38aeb7043f29468fb4f4429376bf3d7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:57.196453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a1712-ee1a-4095-bd1e-f3c8886c2f1f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:57.196604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-11a69ed0-c559-4502-a190-efe15b398443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:57.235894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:57.304119 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e31af3-c502-4168-af3d-a22f36bc95fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:57.305296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e31af3-c502-4168-af3d-a22f36bc95fd admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:57.305892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e31af3-c502-4168-af3d-a22f36bc95fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:57.306529 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e31af3-c502-4168-af3d-a22f36bc95fd admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:57.308071 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e31af3-c502-4168-af3d-a22f36bc95fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:57.310802 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e31af3-c502-4168-af3d-a22f36bc95fd admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:57.364102 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 862/1696] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:04:56 2022] POST /identity/v3/users => generated 283 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:57.373850 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b5b54-edc1-4143-a131-89980f04db09 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:57.375990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b5b54-edc1-4143-a131-89980f04db09 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:57.377354 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b5b54-edc1-4143-a131-89980f04db09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:57.445128 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 835/1697] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:57 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:04:57.445128 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a69ed0-c559-4502-a190-efe15b398443 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:57.464879 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4e0b5b54-edc1-4143-a131-89980f04db09 None None] MFA Rules not processed for user `8df66990a3734209a350c8b6768d259f`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:57.480805 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4e0b5b54-edc1-4143-a131-89980f04db09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:57.496640 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a69ed0-c559-4502-a190-efe15b398443 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:57.524041 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094af7ee-fb81-48e9-88ca-8dade55d56c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:57.527488 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094af7ee-fb81-48e9-88ca-8dade55d56c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:57.528007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094af7ee-fb81-48e9-88ca-8dade55d56c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:57.528608 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094af7ee-fb81-48e9-88ca-8dade55d56c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:57.533620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094af7ee-fb81-48e9-88ca-8dade55d56c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:57.541365 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094af7ee-fb81-48e9-88ca-8dade55d56c6 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:57.578531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 836/1698] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:57.582832 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094af7ee-fb81-48e9-88ca-8dade55d56c6 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:57.608690 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094af7ee-fb81-48e9-88ca-8dade55d56c6 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:57.623165 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9672526-abb8-4e60-b3d0-99183f4cd63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:57.624458 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9672526-abb8-4e60-b3d0-99183f4cd63e admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:57.624961 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9672526-abb8-4e60-b3d0-99183f4cd63e admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:57.627039 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9672526-abb8-4e60-b3d0-99183f4cd63e admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:57.628243 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9672526-abb8-4e60-b3d0-99183f4cd63e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:57.630597 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9672526-abb8-4e60-b3d0-99183f4cd63e admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:57.693936 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:57.712508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 863/1699] 10.222.0.86 () {58 vars in 1015 bytes} [Thu May 26 18:04:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:57.714734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b5b54-edc1-4143-a131-89980f04db09 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:57.727875 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 837/1700] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:04:57 2022] POST /identity/v3/projects => generated 367 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:57.737563 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9672526-abb8-4e60-b3d0-99183f4cd63e admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:57.746330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b5b54-edc1-4143-a131-89980f04db09 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:57.823005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ec49b-a7c1-4735-a43f-1fdf01b080e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:57.824081 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ec49b-a7c1-4735-a43f-1fdf01b080e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:57.824535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ec49b-a7c1-4735-a43f-1fdf01b080e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:57.824957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ec49b-a7c1-4735-a43f-1fdf01b080e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:57.825834 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ec49b-a7c1-4735-a43f-1fdf01b080e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:57.827702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ec49b-a7c1-4735-a43f-1fdf01b080e0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:57.864415 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 864/1701] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:57.865638 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ec49b-a7c1-4735-a43f-1fdf01b080e0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:57.873782 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f64ec49b-a7c1-4735-a43f-1fdf01b080e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:58.015329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ec49b-a7c1-4735-a43f-1fdf01b080e0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:58.028556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9672526-abb8-4e60-b3d0-99183f4cd63e admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:58.032272 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c318a-4f95-43ba-9835-72867a389a6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:58.034417 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c318a-4f95-43ba-9835-72867a389a6c service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:58.034720 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c318a-4f95-43ba-9835-72867a389a6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:58.035043 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c318a-4f95-43ba-9835-72867a389a6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:58.041323 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97fad9c-138b-4a0e-b8ad-e903ffda378c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:58.042177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97fad9c-138b-4a0e-b8ad-e903ffda378c admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:58.043664 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97fad9c-138b-4a0e-b8ad-e903ffda378c admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:58.044185 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97fad9c-138b-4a0e-b8ad-e903ffda378c admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:58.045078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97fad9c-138b-4a0e-b8ad-e903ffda378c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:58.050469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:58.057065 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97fad9c-138b-4a0e-b8ad-e903ffda378c admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:58.061668 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c318a-4f95-43ba-9835-72867a389a6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:58.069608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c318a-4f95-43ba-9835-72867a389a6c service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:58.105001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 838/1702] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:04:57 2022] POST /identity/v3/users => generated 290 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:58.109429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97fad9c-138b-4a0e-b8ad-e903ffda378c admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:58.112834 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 865/1703] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:04:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:04:58.115319 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c318a-4f95-43ba-9835-72867a389a6c service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:58.135805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97fad9c-138b-4a0e-b8ad-e903ffda378c admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:58.136061 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c318a-4f95-43ba-9835-72867a389a6c service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:58.141074 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74afaef9-bfd4-43e9-ae61-eebef8aca426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:58.141074 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afaef9-bfd4-43e9-ae61-eebef8aca426 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:58.141494 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afaef9-bfd4-43e9-ae61-eebef8aca426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:58.141494 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afaef9-bfd4-43e9-ae61-eebef8aca426 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:58.142271 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74afaef9-bfd4-43e9-ae61-eebef8aca426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:58.144061 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74afaef9-bfd4-43e9-ae61-eebef8aca426 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:58.146854 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69d69a-8b0b-459d-bdab-a6ec6a2b6908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:58.147918 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69d69a-8b0b-459d-bdab-a6ec6a2b6908 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:58.148151 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69d69a-8b0b-459d-bdab-a6ec6a2b6908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:58.148429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69d69a-8b0b-459d-bdab-a6ec6a2b6908 admin admin] PATH_INFO: `/v3/projects/204f8f384bc54e699f26d5f9ba6bc590/users/54ffdd4c8cfd41319a04db3cebdadf84/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:58.159381 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 866/1704] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:58.160397 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74afaef9-bfd4-43e9-ae61-eebef8aca426 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:58.162394 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-74afaef9-bfd4-43e9-ae61-eebef8aca426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:58.181488 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69d69a-8b0b-459d-bdab-a6ec6a2b6908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=204f8f384bc54e699f26d5f9ba6bc590, user_id=54ffdd4c8cfd41319a04db3cebdadf84, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:58.185043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69d69a-8b0b-459d-bdab-a6ec6a2b6908 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:58.193355 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74afaef9-bfd4-43e9-ae61-eebef8aca426 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:58.201103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d231464-d5d0-4cde-9873-85f2d2d718a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:58.209193 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d231464-d5d0-4cde-9873-85f2d2d718a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:58.211689 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d231464-d5d0-4cde-9873-85f2d2d718a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:58.211987 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d231464-d5d0-4cde-9873-85f2d2d718a2 admin admin] PATH_INFO: `/v3/projects/9c09ef491df74d48b81e8346d6ee228a` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:58.213810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d231464-d5d0-4cde-9873-85f2d2d718a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c09ef491df74d48b81e8346d6ee228a)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:58.230242 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 839/1705] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:04:58 2022] PUT /identity/v3/projects/204f8f384bc54e699f26d5f9ba6bc590/users/54ffdd4c8cfd41319a04db3cebdadf84/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:58.230242 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69d69a-8b0b-459d-bdab-a6ec6a2b6908 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:58.231259 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d231464-d5d0-4cde-9873-85f2d2d718a2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:58.317752 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69d69a-8b0b-459d-bdab-a6ec6a2b6908 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:58.337170 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5d231464-d5d0-4cde-9873-85f2d2d718a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c09ef491df74d48b81e8346d6ee228a', 'request_id': 'req-5d231464-d5d0-4cde-9873-85f2d2d718a2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:58.356364 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5d231464-d5d0-4cde-9873-85f2d2d718a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c09ef491df74d48b81e8346d6ee228a', 'request_id': 'req-5d231464-d5d0-4cde-9873-85f2d2d718a2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:58.368979 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5d231464-d5d0-4cde-9873-85f2d2d718a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c09ef491df74d48b81e8346d6ee228a', 'request_id': 'req-5d231464-d5d0-4cde-9873-85f2d2d718a2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:58.400002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5d231464-d5d0-4cde-9873-85f2d2d718a2 admin admin] The token cache is being invalidate because project 9c09ef491df74d48b81e8346d6ee228a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:04:58.400347 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5d231464-d5d0-4cde-9873-85f2d2d718a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:58.401272 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5d231464-d5d0-4cde-9873-85f2d2d718a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:04:58.409059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 867/1706] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:04:58 2022] DELETE /identity/v3/projects/9c09ef491df74d48b81e8346d6ee228a => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:58.409059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73640ec0-bc6d-4052-82db-677f7c03244d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:58.409059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73640ec0-bc6d-4052-82db-677f7c03244d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:58.409642 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73640ec0-bc6d-4052-82db-677f7c03244d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:58.413562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9b35f-fd38-4ce7-81a1-3338628ab5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:58.416248 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9b35f-fd38-4ce7-81a1-3338628ab5c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:58.416549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9b35f-fd38-4ce7-81a1-3338628ab5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:58.418927 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9b35f-fd38-4ce7-81a1-3338628ab5c5 admin admin] PATH_INFO: `/v3/projects/35b35e4cdd1c4e779afad8b4d0b0ed5d/users/b5fbc2247ca24c40bdc92cf9f7c51afe/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:58.452549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:58.460158 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9b35f-fd38-4ce7-81a1-3338628ab5c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35b35e4cdd1c4e779afad8b4d0b0ed5d, user_id=b5fbc2247ca24c40bdc92cf9f7c51afe, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:58.463019 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9b35f-fd38-4ce7-81a1-3338628ab5c5 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:58.473418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-73640ec0-bc6d-4052-82db-677f7c03244d None None] MFA Rules not processed for user `54ffdd4c8cfd41319a04db3cebdadf84`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:58.476135 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-73640ec0-bc6d-4052-82db-677f7c03244d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:58.495715 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 840/1707] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:04:58 2022] PUT /identity/v3/projects/35b35e4cdd1c4e779afad8b4d0b0ed5d/users/b5fbc2247ca24c40bdc92cf9f7c51afe/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:58.505043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ace6d4-b34b-4d8a-a8bc-76ad37d0bc74 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:58.505337 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ace6d4-b34b-4d8a-a8bc-76ad37d0bc74 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:58.505810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ace6d4-b34b-4d8a-a8bc-76ad37d0bc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:58.547756 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-31ace6d4-b34b-4d8a-a8bc-76ad37d0bc74 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:58.549014 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-31ace6d4-b34b-4d8a-a8bc-76ad37d0bc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:58.669796 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 868/1708] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:58 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:58.669796 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73640ec0-bc6d-4052-82db-677f7c03244d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:58.671911 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-73640ec0-bc6d-4052-82db-677f7c03244d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:58.714708 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 841/1709] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:58.725227 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ace6d4-b34b-4d8a-a8bc-76ad37d0bc74 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:58.725227 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-31ace6d4-b34b-4d8a-a8bc-76ad37d0bc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:58.745752 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ace6d4-b34b-4d8a-a8bc-76ad37d0bc74 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:58.763879 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473196ce-ba44-42d8-9efc-4b4a0f595299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:58.769803 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473196ce-ba44-42d8-9efc-4b4a0f595299 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:58.770213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473196ce-ba44-42d8-9efc-4b4a0f595299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:58.772821 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473196ce-ba44-42d8-9efc-4b4a0f595299 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:58.773617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473196ce-ba44-42d8-9efc-4b4a0f595299 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:58.775780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473196ce-ba44-42d8-9efc-4b4a0f595299 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:58.843180 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 842/1710] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:04:58 2022] POST /identity/v3/projects => generated 365 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:58.843421 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:58.844151 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615e2d3-3164-4c7f-b746-eae97324e31d None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:58.844151 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615e2d3-3164-4c7f-b746-eae97324e31d None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:58.844151 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615e2d3-3164-4c7f-b746-eae97324e31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:58.850921 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73640ec0-bc6d-4052-82db-677f7c03244d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:58.888842 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-f615e2d3-3164-4c7f-b746-eae97324e31d None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:58.891866 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f615e2d3-3164-4c7f-b746-eae97324e31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:58.996311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227986bf-9b23-4f2c-83a4-fb169a4f529c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:58.996641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227986bf-9b23-4f2c-83a4-fb169a4f529c admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:58.996783 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227986bf-9b23-4f2c-83a4-fb169a4f529c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:58.997053 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227986bf-9b23-4f2c-83a4-fb169a4f529c admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:58.997798 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227986bf-9b23-4f2c-83a4-fb169a4f529c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:58.999540 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227986bf-9b23-4f2c-83a4-fb169a4f529c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:59.025976 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 869/1711] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:59.032502 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227986bf-9b23-4f2c-83a4-fb169a4f529c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:59.048530 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 843/1712] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:59.048530 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615e2d3-3164-4c7f-b746-eae97324e31d None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:59.057644 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227986bf-9b23-4f2c-83a4-fb169a4f529c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:59.064965 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615e2d3-3164-4c7f-b746-eae97324e31d None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:59.070608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c053a-e15c-46b4-9dc1-6a67e9b9fa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:59.071317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c053a-e15c-46b4-9dc1-6a67e9b9fa48 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:59.071604 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c053a-e15c-46b4-9dc1-6a67e9b9fa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:59.071884 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c053a-e15c-46b4-9dc1-6a67e9b9fa48 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:59.072589 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c053a-e15c-46b4-9dc1-6a67e9b9fa48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:59.074628 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c053a-e15c-46b4-9dc1-6a67e9b9fa48 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:59.078722 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8d1e2-7c75-4416-95ab-ad6db91a3a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:59.079653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8d1e2-7c75-4416-95ab-ad6db91a3a2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:59.081556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8d1e2-7c75-4416-95ab-ad6db91a3a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:59.081556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8d1e2-7c75-4416-95ab-ad6db91a3a2f admin admin] PATH_INFO: `/v3/projects/35b35e4cdd1c4e779afad8b4d0b0ed5d/users/b5fbc2247ca24c40bdc92cf9f7c51afe/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:59.086563 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8d1e2-7c75-4416-95ab-ad6db91a3a2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=35b35e4cdd1c4e779afad8b4d0b0ed5d, user_id=b5fbc2247ca24c40bdc92cf9f7c51afe, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:59.089308 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8d1e2-7c75-4416-95ab-ad6db91a3a2f admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:59.132884 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 844/1713] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:04:59 2022] PUT /identity/v3/projects/35b35e4cdd1c4e779afad8b4d0b0ed5d/users/b5fbc2247ca24c40bdc92cf9f7c51afe/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:59.132884 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8d1e2-7c75-4416-95ab-ad6db91a3a2f admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:59.133255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 870/1714] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:04:59 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:04:59.134180 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c053a-e15c-46b4-9dc1-6a67e9b9fa48 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:59.159308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c053a-e15c-46b4-9dc1-6a67e9b9fa48 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:59.352574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8575f4-9e96-4bdc-b46b-9a8ab7edde3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:59.353332 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8575f4-9e96-4bdc-b46b-9a8ab7edde3d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:59.353548 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8575f4-9e96-4bdc-b46b-9a8ab7edde3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:59.353825 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8575f4-9e96-4bdc-b46b-9a8ab7edde3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:59.354443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:04:59.355283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8575f4-9e96-4bdc-b46b-9a8ab7edde3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:59.359068 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8575f4-9e96-4bdc-b46b-9a8ab7edde3d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:59.374533 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8d1e2-7c75-4416-95ab-ad6db91a3a2f admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:59.388648 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942871d-7365-4a56-8ef6-f485a9d7f2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:59.390307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942871d-7365-4a56-8ef6-f485a9d7f2ab admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:59.390970 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942871d-7365-4a56-8ef6-f485a9d7f2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:59.394864 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942871d-7365-4a56-8ef6-f485a9d7f2ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:59.398853 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8942871d-7365-4a56-8ef6-f485a9d7f2ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:04:59.402642 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8942871d-7365-4a56-8ef6-f485a9d7f2ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:59.410988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 871/1715] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:04:59.411242 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8942871d-7365-4a56-8ef6-f485a9d7f2ab admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:59.421593 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70598dcd-bea6-4301-97f7-7f3db11d0bbe None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:59.421907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70598dcd-bea6-4301-97f7-7f3db11d0bbe None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:59.422490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70598dcd-bea6-4301-97f7-7f3db11d0bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:59.481664 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 845/1716] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:04:59.492741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942871d-7365-4a56-8ef6-f485a9d7f2ab admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:59.526308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-70598dcd-bea6-4301-97f7-7f3db11d0bbe None None] MFA Rules not processed for user `b5fbc2247ca24c40bdc92cf9f7c51afe`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:04:59.528134 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-70598dcd-bea6-4301-97f7-7f3db11d0bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:04:59.546950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942871d-7365-4a56-8ef6-f485a9d7f2ab admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:04:59.586505 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b69636-9b01-4701-baf3-624298085732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:04:59.594047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b69636-9b01-4701-baf3-624298085732 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:04:59.594944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b69636-9b01-4701-baf3-624298085732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:04:59.595231 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b69636-9b01-4701-baf3-624298085732 admin admin] PATH_INFO: `/v3/projects/0ed06a764de944f09e6656b70280f04d/users/bd92aab18b924c738566dcd9dfe08c24/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:04:59.695162 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b69636-9b01-4701-baf3-624298085732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ed06a764de944f09e6656b70280f04d, user_id=bd92aab18b924c738566dcd9dfe08c24, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:04:59.708763 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b69636-9b01-4701-baf3-624298085732 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:04:59.762883 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 846/1717] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:04:59 2022] PUT /identity/v3/projects/0ed06a764de944f09e6656b70280f04d/users/bd92aab18b924c738566dcd9dfe08c24/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:04:59.763837 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b69636-9b01-4701-baf3-624298085732 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:04:59.984274 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 872/1718] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:04:59 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 573 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:04:59.986130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70598dcd-bea6-4301-97f7-7f3db11d0bbe None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:00.011874 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:00.021831 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b69636-9b01-4701-baf3-624298085732 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:00.027766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70598dcd-bea6-4301-97f7-7f3db11d0bbe None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:00.039623 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6790e-08c4-47fa-8851-298d68b46a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:00.040385 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6790e-08c4-47fa-8851-298d68b46a49 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:00.040682 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6790e-08c4-47fa-8851-298d68b46a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:00.041069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6790e-08c4-47fa-8851-298d68b46a49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:00.041821 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb6790e-08c4-47fa-8851-298d68b46a49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:00.044504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb6790e-08c4-47fa-8851-298d68b46a49 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:00.045349 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7d756-d08b-4fc4-be5c-5aec50bcd42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:00.052039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7d756-d08b-4fc4-be5c-5aec50bcd42e admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:00.052784 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7d756-d08b-4fc4-be5c-5aec50bcd42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:00.053196 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7d756-d08b-4fc4-be5c-5aec50bcd42e admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:00.054111 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7d756-d08b-4fc4-be5c-5aec50bcd42e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:00.058634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7d756-d08b-4fc4-be5c-5aec50bcd42e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:00.113907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 873/1719] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:00.113907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7d756-d08b-4fc4-be5c-5aec50bcd42e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:00.113907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c3f7d756-d08b-4fc4-be5c-5aec50bcd42e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:00.133022 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 847/1720] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:04:59 2022] POST /identity/v3/projects => generated 403 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:00.134058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6790e-08c4-47fa-8851-298d68b46a49 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:00.306371 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7d756-d08b-4fc4-be5c-5aec50bcd42e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:00.325610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6790e-08c4-47fa-8851-298d68b46a49 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:00.335956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14951dd-78ac-485c-a4f4-180c9a8ed25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:00.338224 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14951dd-78ac-485c-a4f4-180c9a8ed25b admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:00.339913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14951dd-78ac-485c-a4f4-180c9a8ed25b admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:00.341981 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14951dd-78ac-485c-a4f4-180c9a8ed25b admin admin] PATH_INFO: `/v3/projects/0ed06a764de944f09e6656b70280f04d/users/bd92aab18b924c738566dcd9dfe08c24/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:00.347556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14951dd-78ac-485c-a4f4-180c9a8ed25b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ed06a764de944f09e6656b70280f04d, user_id=bd92aab18b924c738566dcd9dfe08c24, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:00.353173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14951dd-78ac-485c-a4f4-180c9a8ed25b admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:00.401236 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 848/1721] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:00 2022] PUT /identity/v3/projects/0ed06a764de944f09e6656b70280f04d/users/bd92aab18b924c738566dcd9dfe08c24/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:00.402473 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14951dd-78ac-485c-a4f4-180c9a8ed25b admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:00.481412 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d54cf9-6e1f-4e7c-b580-9cd3c961c66f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:00.486358 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d54cf9-6e1f-4e7c-b580-9cd3c961c66f service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:00.487617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d54cf9-6e1f-4e7c-b580-9cd3c961c66f service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:00.489417 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d54cf9-6e1f-4e7c-b580-9cd3c961c66f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:00.591318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:00.643524 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14951dd-78ac-485c-a4f4-180c9a8ed25b admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:00.664566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cfe27b-aa90-4bec-9b11-e49bb4d362fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:00.667268 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfe27b-aa90-4bec-9b11-e49bb4d362fd admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:00.667872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfe27b-aa90-4bec-9b11-e49bb4d362fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:00.668923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfe27b-aa90-4bec-9b11-e49bb4d362fd admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:00.672214 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cfe27b-aa90-4bec-9b11-e49bb4d362fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:00.678882 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cfe27b-aa90-4bec-9b11-e49bb4d362fd admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:00.705589 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d54cf9-6e1f-4e7c-b580-9cd3c961c66f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:00.711607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d54cf9-6e1f-4e7c-b580-9cd3c961c66f service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:00.767822 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 849/1722] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:05:00 2022] POST /identity/v3/users => generated 302 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:00.771262 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454aa034-75aa-441f-a30c-8514651f817c None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:00.772640 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454aa034-75aa-441f-a30c-8514651f817c None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:00.773755 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 874/1723] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:00 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 669 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:00.789532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454aa034-75aa-441f-a30c-8514651f817c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:00.796028 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d54cf9-6e1f-4e7c-b580-9cd3c961c66f service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:00.857980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d54cf9-6e1f-4e7c-b580-9cd3c961c66f service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:00.889896 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af769ff5-1804-44e4-a8a5-aff5edb6492d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:00.895793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af769ff5-1804-44e4-a8a5-aff5edb6492d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:00.897161 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af769ff5-1804-44e4-a8a5-aff5edb6492d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:00.898457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af769ff5-1804-44e4-a8a5-aff5edb6492d admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:00.904306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af769ff5-1804-44e4-a8a5-aff5edb6492d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:00.913617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af769ff5-1804-44e4-a8a5-aff5edb6492d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:00.953275 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-454aa034-75aa-441f-a30c-8514651f817c None None] MFA Rules not processed for user `bd92aab18b924c738566dcd9dfe08c24`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:00.961346 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-454aa034-75aa-441f-a30c-8514651f817c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:00.972987 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 875/1724] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:00.974075 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af769ff5-1804-44e4-a8a5-aff5edb6492d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:00.987449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-af769ff5-1804-44e4-a8a5-aff5edb6492d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:01.228431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 850/1725] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:00 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:01.229695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454aa034-75aa-441f-a30c-8514651f817c None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:01.242246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:01.256452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af769ff5-1804-44e4-a8a5-aff5edb6492d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:01.275623 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba801e4-e3d5-494e-8d8f-dacfdb01100b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:01.276312 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba801e4-e3d5-494e-8d8f-dacfdb01100b service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:01.276591 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba801e4-e3d5-494e-8d8f-dacfdb01100b service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:01.276866 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba801e4-e3d5-494e-8d8f-dacfdb01100b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:01.279191 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454aa034-75aa-441f-a30c-8514651f817c None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:01.286626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fba801e4-e3d5-494e-8d8f-dacfdb01100b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:01.307697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173d1d7-9eba-4fbf-9289-a9b375559a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:01.308398 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173d1d7-9eba-4fbf-9289-a9b375559a80 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:01.308959 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173d1d7-9eba-4fbf-9289-a9b375559a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:01.308959 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173d1d7-9eba-4fbf-9289-a9b375559a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:01.309728 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a173d1d7-9eba-4fbf-9289-a9b375559a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:01.317697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a173d1d7-9eba-4fbf-9289-a9b375559a80 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:01.359488 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 851/1726] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:01.362698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173d1d7-9eba-4fbf-9289-a9b375559a80 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:01.387401 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173d1d7-9eba-4fbf-9289-a9b375559a80 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:01.404767 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c13c5e-9989-49d8-a666-5f63b8c073b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:01.406218 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c13c5e-9989-49d8-a666-5f63b8c073b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:01.407940 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c13c5e-9989-49d8-a666-5f63b8c073b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:01.409603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c13c5e-9989-49d8-a666-5f63b8c073b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:01.462970 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c13c5e-9989-49d8-a666-5f63b8c073b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:01.465422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c13c5e-9989-49d8-a666-5f63b8c073b1 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:01.492451 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 852/1727] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:01 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:01.503227 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c13c5e-9989-49d8-a666-5f63b8c073b1 service cinder] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:01.556978 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c13c5e-9989-49d8-a666-5f63b8c073b1 service cinder] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:01.565246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba801e4-e3d5-494e-8d8f-dacfdb01100b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:01.568819 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba801e4-e3d5-494e-8d8f-dacfdb01100b service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:01.572153 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3d253-4b74-4516-acfe-c0e2810cbc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:01.574245 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3d253-4b74-4516-acfe-c0e2810cbc63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:01.575513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3d253-4b74-4516-acfe-c0e2810cbc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:01.575863 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3d253-4b74-4516-acfe-c0e2810cbc63 admin admin] PATH_INFO: `/v3/projects/2672a9a24f72494b900035c0b231c769/users/4624076c5b2b446e8ebba127d3a585da/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:01.608447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 876/1728] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:00 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:01.630613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3d253-4b74-4516-acfe-c0e2810cbc63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2672a9a24f72494b900035c0b231c769, user_id=4624076c5b2b446e8ebba127d3a585da, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:01.635091 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3d253-4b74-4516-acfe-c0e2810cbc63 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:01.689935 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 853/1729] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:01 2022] PUT /identity/v3/projects/2672a9a24f72494b900035c0b231c769/users/4624076c5b2b446e8ebba127d3a585da/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:01.690996 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978c319-174d-4308-827a-fb82a7814847 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:01.691317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978c319-174d-4308-827a-fb82a7814847 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:01.691696 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978c319-174d-4308-827a-fb82a7814847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:01.762550 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3d253-4b74-4516-acfe-c0e2810cbc63 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:01.806838 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4978c319-174d-4308-827a-fb82a7814847 None None] MFA Rules not processed for user `4624076c5b2b446e8ebba127d3a585da`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:01.811540 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4978c319-174d-4308-827a-fb82a7814847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:02.045427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3d253-4b74-4516-acfe-c0e2810cbc63 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:02.058707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608981ac-db2e-4ab7-9f25-d5d4bc40526c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:02.059856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608981ac-db2e-4ab7-9f25-d5d4bc40526c service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:02.060870 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608981ac-db2e-4ab7-9f25-d5d4bc40526c service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:02.062298 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608981ac-db2e-4ab7-9f25-d5d4bc40526c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:02.145282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 877/1730] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:02.146971 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893407da-7b8c-4e88-9f12-b58057db23fe None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:02.147562 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893407da-7b8c-4e88-9f12-b58057db23fe None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:02.147955 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893407da-7b8c-4e88-9f12-b58057db23fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:02.164289 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:02.268166 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-893407da-7b8c-4e88-9f12-b58057db23fe None None] MFA Rules not processed for user `dfbe23f3a0294148a88f4758d8470665`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:02.274473 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-893407da-7b8c-4e88-9f12-b58057db23fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:02.278076 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608981ac-db2e-4ab7-9f25-d5d4bc40526c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:02.280812 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608981ac-db2e-4ab7-9f25-d5d4bc40526c service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:02.312973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 854/1731] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:01 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:02.314931 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608981ac-db2e-4ab7-9f25-d5d4bc40526c service cinder] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:02.555258 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 878/1732] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:02.576562 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48137e8e-0420-4a1d-89f1-998896a4b63a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:02.576854 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48137e8e-0420-4a1d-89f1-998896a4b63a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:02.577187 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48137e8e-0420-4a1d-89f1-998896a4b63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:02.629734 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608981ac-db2e-4ab7-9f25-d5d4bc40526c service cinder] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:02.648268 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75e223-f8f0-4098-b825-47cc9c6df3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:02.648555 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75e223-f8f0-4098-b825-47cc9c6df3ec admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:02.649260 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75e223-f8f0-4098-b825-47cc9c6df3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:02.649932 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75e223-f8f0-4098-b825-47cc9c6df3ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:02.651142 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75e223-f8f0-4098-b825-47cc9c6df3ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:02.656452 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75e223-f8f0-4098-b825-47cc9c6df3ec admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:02.753877 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 855/1733] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:05:02 2022] POST /identity/v3/projects => generated 405 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:02.763459 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75e223-f8f0-4098-b825-47cc9c6df3ec admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:02.773799 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-1e75e223-f8f0-4098-b825-47cc9c6df3ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:02.780390 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-48137e8e-0420-4a1d-89f1-998896a4b63a None None] MFA Rules not processed for user `e5e66ed4a9b346a68bb70206027946a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:02.782691 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-48137e8e-0420-4a1d-89f1-998896a4b63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:02.865666 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:03.098872 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 879/1734] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:02 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 542 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:03.100535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5ff58-9fdb-41cb-adf9-65a31bbfd019 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:03.104082 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5ff58-9fdb-41cb-adf9-65a31bbfd019 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:03.104645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5ff58-9fdb-41cb-adf9-65a31bbfd019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:03.138824 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75e223-f8f0-4098-b825-47cc9c6df3ec admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:03.169296 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9c379-475d-4266-b8f2-eee7cb84aa40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:03.169963 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9c379-475d-4266-b8f2-eee7cb84aa40 service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:03.170280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9c379-475d-4266-b8f2-eee7cb84aa40 service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:03.170591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9c379-475d-4266-b8f2-eee7cb84aa40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:03.256223 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c0a5ff58-9fdb-41cb-adf9-65a31bbfd019 None None] MFA Rules not processed for user `61f6dbd4328e459eb7d4f029551648bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:03.268213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c0a5ff58-9fdb-41cb-adf9-65a31bbfd019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:03.325703 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:03.355518 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd9c379-475d-4266-b8f2-eee7cb84aa40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:03.362159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd9c379-475d-4266-b8f2-eee7cb84aa40 service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:03.396534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 856/1735] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:03.397629 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9c379-475d-4266-b8f2-eee7cb84aa40 service nova] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:03.663208 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9c379-475d-4266-b8f2-eee7cb84aa40 service nova] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:03.686241 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 880/1736] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 587 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:03.688867 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db6547-3e17-4057-ba7c-c50f67ab5f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:03.702821 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db6547-3e17-4057-ba7c-c50f67ab5f6a admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:03.707763 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db6547-3e17-4057-ba7c-c50f67ab5f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:03.709402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db6547-3e17-4057-ba7c-c50f67ab5f6a admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:03.710094 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122399a0-2377-4de7-b13d-991de38fe505 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:03.712115 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122399a0-2377-4de7-b13d-991de38fe505 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:03.713227 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db6547-3e17-4057-ba7c-c50f67ab5f6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:03.713598 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122399a0-2377-4de7-b13d-991de38fe505 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:03.717230 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db6547-3e17-4057-ba7c-c50f67ab5f6a admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:03.721419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 881/1737] 10.222.0.76 () {54 vars in 921 bytes} [Thu May 26 18:05:03 2022] GET /identity => generated 270 bytes in 31 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:03.728495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e0ceb-47a6-4af2-86c8-6ba1e77500e4 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:03.730964 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e0ceb-47a6-4af2-86c8-6ba1e77500e4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:03.732394 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e0ceb-47a6-4af2-86c8-6ba1e77500e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:03.801327 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 857/1738] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:05:03 2022] POST /identity/v3/users => generated 309 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:03.802728 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db6547-3e17-4057-ba7c-c50f67ab5f6a admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:03.822745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c29e0ceb-47a6-4af2-86c8-6ba1e77500e4 None None] MFA Rules not processed for user `61f6dbd4328e459eb7d4f029551648bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:03.826103 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db6547-3e17-4057-ba7c-c50f67ab5f6a admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:03.829175 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c29e0ceb-47a6-4af2-86c8-6ba1e77500e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:03.852969 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424c2bb-b2d9-41ce-b840-6266a9ed0133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:03.854044 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424c2bb-b2d9-41ce-b840-6266a9ed0133 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:03.855195 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424c2bb-b2d9-41ce-b840-6266a9ed0133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:03.855706 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424c2bb-b2d9-41ce-b840-6266a9ed0133 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:03.856681 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3424c2bb-b2d9-41ce-b840-6266a9ed0133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:03.860954 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3424c2bb-b2d9-41ce-b840-6266a9ed0133 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:03.885873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 882/1739] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:03.895565 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83ecd1-98cf-4001-94a5-9d70c5a9d687 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:03.897504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83ecd1-98cf-4001-94a5-9d70c5a9d687 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:03.899522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83ecd1-98cf-4001-94a5-9d70c5a9d687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:03.966561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 858/1740] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:03.966561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424c2bb-b2d9-41ce-b840-6266a9ed0133 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:03.973984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-3424c2bb-b2d9-41ce-b840-6266a9ed0133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:04.028427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424c2bb-b2d9-41ce-b840-6266a9ed0133 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:04.039949 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5e83ecd1-98cf-4001-94a5-9d70c5a9d687 None None] MFA Rules not processed for user `1899c317083d40f5b490cd9eb63fe521`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:04.041917 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5e83ecd1-98cf-4001-94a5-9d70c5a9d687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:04.042428 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae3fbb-7948-4ec6-a829-d1ffb83ae665 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:04.046321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae3fbb-7948-4ec6-a829-d1ffb83ae665 service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:04.046948 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae3fbb-7948-4ec6-a829-d1ffb83ae665 service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:04.047964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae3fbb-7948-4ec6-a829-d1ffb83ae665 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:04.097906 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae3fbb-7948-4ec6-a829-d1ffb83ae665 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:04.099723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae3fbb-7948-4ec6-a829-d1ffb83ae665 service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:04.134177 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 883/1741] 10.222.0.76 () {58 vars in 1018 bytes} [Thu May 26 18:05:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:04.135649 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83ecd1-98cf-4001-94a5-9d70c5a9d687 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:04.167220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 859/1742] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:04.200425 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83ecd1-98cf-4001-94a5-9d70c5a9d687 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:04.238285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8982fe64-d8a1-4366-b64b-c2eaf1d38262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:04.242309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982fe64-d8a1-4366-b64b-c2eaf1d38262 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:04.246512 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982fe64-d8a1-4366-b64b-c2eaf1d38262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:04.248751 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982fe64-d8a1-4366-b64b-c2eaf1d38262 admin admin] PATH_INFO: `/v3/projects/1d57dcb2aae7403fb0fce84c93f5b763/users/9a263de1d1074abf8756b9e07b9e9f1d/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:04.333575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8982fe64-d8a1-4366-b64b-c2eaf1d38262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d57dcb2aae7403fb0fce84c93f5b763, user_id=9a263de1d1074abf8756b9e07b9e9f1d, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:04.341844 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8982fe64-d8a1-4366-b64b-c2eaf1d38262 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:04.377494 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 884/1743] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:04 2022] PUT /identity/v3/projects/1d57dcb2aae7403fb0fce84c93f5b763/users/9a263de1d1074abf8756b9e07b9e9f1d/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:04.382678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae3fbb-7948-4ec6-a829-d1ffb83ae665 service nova] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:04.396397 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8982fe64-d8a1-4366-b64b-c2eaf1d38262 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:04.411591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae3fbb-7948-4ec6-a829-d1ffb83ae665 service nova] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:04.560316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:04.620294 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8982fe64-d8a1-4366-b64b-c2eaf1d38262 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:04.661307 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba442b23-663f-4fff-a991-1ee8c52c5be6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:04.672561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba442b23-663f-4fff-a991-1ee8c52c5be6 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:04.672940 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba442b23-663f-4fff-a991-1ee8c52c5be6 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:04.673396 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba442b23-663f-4fff-a991-1ee8c52c5be6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:04.717851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba442b23-663f-4fff-a991-1ee8c52c5be6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:04.719686 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba442b23-663f-4fff-a991-1ee8c52c5be6 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:04.749028 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bc911-a4b8-4f9a-b3eb-7fbb9ecc84ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:04.752932 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bc911-a4b8-4f9a-b3eb-7fbb9ecc84ef admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:04.753236 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bc911-a4b8-4f9a-b3eb-7fbb9ecc84ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:04.753516 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bc911-a4b8-4f9a-b3eb-7fbb9ecc84ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:04.760819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78bc911-a4b8-4f9a-b3eb-7fbb9ecc84ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:04.763436 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 885/1744] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:05:04.767678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78bc911-a4b8-4f9a-b3eb-7fbb9ecc84ef admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:04.823776 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 860/1745] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:04.833712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba442b23-663f-4fff-a991-1ee8c52c5be6 service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:04.860787 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba442b23-663f-4fff-a991-1ee8c52c5be6 service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:04.882214 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf89343-a4c7-4c41-b2ee-d53d7b2014ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:04.885452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf89343-a4c7-4c41-b2ee-d53d7b2014ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:04.889153 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf89343-a4c7-4c41-b2ee-d53d7b2014ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:04.889493 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf89343-a4c7-4c41-b2ee-d53d7b2014ac admin admin] PATH_INFO: `/v3/projects/1d57dcb2aae7403fb0fce84c93f5b763/users/9a263de1d1074abf8756b9e07b9e9f1d/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:04.898739 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf89343-a4c7-4c41-b2ee-d53d7b2014ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d57dcb2aae7403fb0fce84c93f5b763, user_id=9a263de1d1074abf8756b9e07b9e9f1d, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:04.912169 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf89343-a4c7-4c41-b2ee-d53d7b2014ac admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:04.964135 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 886/1746] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:04 2022] PUT /identity/v3/projects/1d57dcb2aae7403fb0fce84c93f5b763/users/9a263de1d1074abf8756b9e07b9e9f1d/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:04.975796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cd8f3-1e69-466b-ae7d-abc748de2fac None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:04.976034 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cd8f3-1e69-466b-ae7d-abc748de2fac None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:04.976271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cd8f3-1e69-466b-ae7d-abc748de2fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:05.105350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-948cd8f3-1e69-466b-ae7d-abc748de2fac None None] MFA Rules not processed for user `9a263de1d1074abf8756b9e07b9e9f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:05.111501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-948cd8f3-1e69-466b-ae7d-abc748de2fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:05.369684 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:05.512609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 861/1747] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:04 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 538 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:05:05.566923 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf89343-a4c7-4c41-b2ee-d53d7b2014ac admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:05.833568 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf89343-a4c7-4c41-b2ee-d53d7b2014ac admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:05.842220 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6de69-7707-4687-87ac-48d33ae0c6a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:05.842637 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6de69-7707-4687-87ac-48d33ae0c6a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:05.843025 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6de69-7707-4687-87ac-48d33ae0c6a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:05.843230 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6de69-7707-4687-87ac-48d33ae0c6a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:05.867781 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6de69-7707-4687-87ac-48d33ae0c6a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:05.869413 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6de69-7707-4687-87ac-48d33ae0c6a1 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:05.904737 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 887/1748] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:05 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:06.272349 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948cd8f3-1e69-466b-ae7d-abc748de2fac None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:06.439808 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948cd8f3-1e69-466b-ae7d-abc748de2fac None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:06.453234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a99f2-4972-45d2-9066-e265999f9fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:06.454441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a99f2-4972-45d2-9066-e265999f9fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:06.454894 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a99f2-4972-45d2-9066-e265999f9fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:06.455286 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a99f2-4972-45d2-9066-e265999f9fa1 admin admin] PATH_INFO: `/v3/users/4624076c5b2b446e8ebba127d3a585da` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:06.457081 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a99f2-4972-45d2-9066-e265999f9fa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4624076c5b2b446e8ebba127d3a585da)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:06.466802 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a99f2-4972-45d2-9066-e265999f9fa1 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:06.673721 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-315a99f2-4972-45d2-9066-e265999f9fa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4624076c5b2b446e8ebba127d3a585da'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:06.704803 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-315a99f2-4972-45d2-9066-e265999f9fa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4624076c5b2b446e8ebba127d3a585da'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:06.705438 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-315a99f2-4972-45d2-9066-e265999f9fa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4624076c5b2b446e8ebba127d3a585da'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:06.706129 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-315a99f2-4972-45d2-9066-e265999f9fa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4624076c5b2b446e8ebba127d3a585da'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:06.755031 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-315a99f2-4972-45d2-9066-e265999f9fa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4624076c5b2b446e8ebba127d3a585da'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:06.877987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 862/1749] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:05:06 2022] DELETE /identity/v3/users/4624076c5b2b446e8ebba127d3a585da => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:06.884231 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78074a71-2c62-499b-9cbb-c65ce8160550 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:06.884913 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78074a71-2c62-499b-9cbb-c65ce8160550 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:06.884913 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78074a71-2c62-499b-9cbb-c65ce8160550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:06.932989 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-78074a71-2c62-499b-9cbb-c65ce8160550 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:06.934756 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-78074a71-2c62-499b-9cbb-c65ce8160550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:07.146826 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 888/1750] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:07.164410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a99f2-4972-45d2-9066-e265999f9fa1 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:07.166077 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-315a99f2-4972-45d2-9066-e265999f9fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:07.440351 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a99f2-4972-45d2-9066-e265999f9fa1 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:07.448581 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f7253-52df-46cd-a1db-f41f5fc51ddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:07.449662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f7253-52df-46cd-a1db-f41f5fc51ddc service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:07.450124 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f7253-52df-46cd-a1db-f41f5fc51ddc service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:07.451006 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f7253-52df-46cd-a1db-f41f5fc51ddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:07.493240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f7253-52df-46cd-a1db-f41f5fc51ddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:07.496132 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f7253-52df-46cd-a1db-f41f5fc51ddc service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:07.506131 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:07.539492 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 863/1751] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:08.117006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db080227-d3dd-4aa6-b84b-77dccde891e7 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:08.117578 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db080227-d3dd-4aa6-b84b-77dccde891e7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:08.117759 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db080227-d3dd-4aa6-b84b-77dccde891e7 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:08.119576 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 889/1752] 10.222.0.76 () {54 vars in 921 bytes} [Thu May 26 18:05:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:08.410898 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c38b12-3fbc-480a-887b-a8c8c390d6fc None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:08.411603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c38b12-3fbc-480a-887b-a8c8c390d6fc None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:08.412053 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c38b12-3fbc-480a-887b-a8c8c390d6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:08.470832 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-28c38b12-3fbc-480a-887b-a8c8c390d6fc None None] MFA Rules not processed for user `596f126382e3440fb3cf7c357eef889f`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:08.476712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-28c38b12-3fbc-480a-887b-a8c8c390d6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:08.660915 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db080227-d3dd-4aa6-b84b-77dccde891e7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:08.663373 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-db080227-d3dd-4aa6-b84b-77dccde891e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:08.699881 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db080227-d3dd-4aa6-b84b-77dccde891e7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:08.716421 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08aa3b-01e7-4fa9-a84b-98e3fe312dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:08.718551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08aa3b-01e7-4fa9-a84b-98e3fe312dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:08.719106 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08aa3b-01e7-4fa9-a84b-98e3fe312dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:08.720208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08aa3b-01e7-4fa9-a84b-98e3fe312dbe admin admin] PATH_INFO: `/v3/users/38aeb7043f29468fb4f4429376bf3d7b` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:08.723297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08aa3b-01e7-4fa9-a84b-98e3fe312dbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=38aeb7043f29468fb4f4429376bf3d7b)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:08.726523 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08aa3b-01e7-4fa9-a84b-98e3fe312dbe admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:08.773867 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 864/1753] 10.222.0.76 () {58 vars in 1018 bytes} [Thu May 26 18:05:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:05:08.869117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c38b12-3fbc-480a-887b-a8c8c390d6fc None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:08.869801 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2a08aa3b-01e7-4fa9-a84b-98e3fe312dbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38aeb7043f29468fb4f4429376bf3d7b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:08.871382 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-28c38b12-3fbc-480a-887b-a8c8c390d6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:08.892908 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2a08aa3b-01e7-4fa9-a84b-98e3fe312dbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38aeb7043f29468fb4f4429376bf3d7b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:08.904431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c38b12-3fbc-480a-887b-a8c8c390d6fc None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:08.919419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8567b-bece-4095-a1fd-3e21b35cf6dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '596f126382e3440fb3cf7c357eef889f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:08.922837 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8567b-bece-4095-a1fd-3e21b35cf6dc service placement] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:08.924794 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8567b-bece-4095-a1fd-3e21b35cf6dc service placement] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:08.927703 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8567b-bece-4095-a1fd-3e21b35cf6dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:08.932598 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:08.937491 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2a08aa3b-01e7-4fa9-a84b-98e3fe312dbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38aeb7043f29468fb4f4429376bf3d7b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:08.940958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2a08aa3b-01e7-4fa9-a84b-98e3fe312dbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38aeb7043f29468fb4f4429376bf3d7b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:08.944331 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2a08aa3b-01e7-4fa9-a84b-98e3fe312dbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38aeb7043f29468fb4f4429376bf3d7b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:08.971088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 890/1754] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:05:08 2022] DELETE /identity/v3/users/38aeb7043f29468fb4f4429376bf3d7b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:08.991081 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa28789-7fa3-42f0-8875-90823864e836 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:08.991627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa28789-7fa3-42f0-8875-90823864e836 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:08.992057 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa28789-7fa3-42f0-8875-90823864e836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:08.998825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8567b-bece-4095-a1fd-3e21b35cf6dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:09.003096 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8567b-bece-4095-a1fd-3e21b35cf6dc service placement] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:09.036893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6fa28789-7fa3-42f0-8875-90823864e836 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:09.038458 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6fa28789-7fa3-42f0-8875-90823864e836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:09.048528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 865/1755] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:09.154156 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 891/1756] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:09.178237 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8567b-bece-4095-a1fd-3e21b35cf6dc service placement] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:09.186693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa28789-7fa3-42f0-8875-90823864e836 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:09.189920 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6fa28789-7fa3-42f0-8875-90823864e836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:09.199719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8567b-bece-4095-a1fd-3e21b35cf6dc service placement] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:09.221125 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa28789-7fa3-42f0-8875-90823864e836 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:09.238645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258bebe-a252-44b9-98c9-1010c77e5d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:09.239629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258bebe-a252-44b9-98c9-1010c77e5d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:09.240592 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258bebe-a252-44b9-98c9-1010c77e5d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:09.241147 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258bebe-a252-44b9-98c9-1010c77e5d0e admin admin] PATH_INFO: `/v3/projects/2672a9a24f72494b900035c0b231c769` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:09.245444 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258bebe-a252-44b9-98c9-1010c77e5d0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2672a9a24f72494b900035c0b231c769)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:09.248163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258bebe-a252-44b9-98c9-1010c77e5d0e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:09.324435 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de9a25-59fe-4384-865d-2b63246f283c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:09.327242 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de9a25-59fe-4384-865d-2b63246f283c service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:09.329114 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de9a25-59fe-4384-865d-2b63246f283c service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:09.329529 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de9a25-59fe-4384-865d-2b63246f283c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:09.354762 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de9a25-59fe-4384-865d-2b63246f283c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:09.356962 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de9a25-59fe-4384-865d-2b63246f283c service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:09.363153 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1258bebe-a252-44b9-98c9-1010c77e5d0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2672a9a24f72494b900035c0b231c769', 'request_id': 'req-1258bebe-a252-44b9-98c9-1010c77e5d0e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:09.377161 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1258bebe-a252-44b9-98c9-1010c77e5d0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2672a9a24f72494b900035c0b231c769', 'request_id': 'req-1258bebe-a252-44b9-98c9-1010c77e5d0e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:09.385977 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 866/1757] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:09.391973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1258bebe-a252-44b9-98c9-1010c77e5d0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2672a9a24f72494b900035c0b231c769', 'request_id': 'req-1258bebe-a252-44b9-98c9-1010c77e5d0e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:09.410984 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1258bebe-a252-44b9-98c9-1010c77e5d0e admin admin] The token cache is being invalidate because project 2672a9a24f72494b900035c0b231c769 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:05:09.411300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1258bebe-a252-44b9-98c9-1010c77e5d0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:09.412503 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1258bebe-a252-44b9-98c9-1010c77e5d0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:09.414766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 892/1758] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:05:09 2022] DELETE /identity/v3/projects/2672a9a24f72494b900035c0b231c769 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:09.418736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de9a25-59fe-4384-865d-2b63246f283c service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:09.441916 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de9a25-59fe-4384-865d-2b63246f283c service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:09.528760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f390cbf-e3fd-49b6-bab8-b5dad081c2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:09.531142 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f390cbf-e3fd-49b6-bab8-b5dad081c2dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:09.532292 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f390cbf-e3fd-49b6-bab8-b5dad081c2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:09.532869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f390cbf-e3fd-49b6-bab8-b5dad081c2dd admin admin] PATH_INFO: `/v3/users/9a263de1d1074abf8756b9e07b9e9f1d` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:09.534653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f390cbf-e3fd-49b6-bab8-b5dad081c2dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a263de1d1074abf8756b9e07b9e9f1d)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:09.536836 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f390cbf-e3fd-49b6-bab8-b5dad081c2dd admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:09.574888 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:09.611031 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2f390cbf-e3fd-49b6-bab8-b5dad081c2dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a263de1d1074abf8756b9e07b9e9f1d'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:09.635624 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2f390cbf-e3fd-49b6-bab8-b5dad081c2dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a263de1d1074abf8756b9e07b9e9f1d'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:09.638780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2f390cbf-e3fd-49b6-bab8-b5dad081c2dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a263de1d1074abf8756b9e07b9e9f1d'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:09.643107 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2f390cbf-e3fd-49b6-bab8-b5dad081c2dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a263de1d1074abf8756b9e07b9e9f1d'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:09.668068 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2f390cbf-e3fd-49b6-bab8-b5dad081c2dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a263de1d1074abf8756b9e07b9e9f1d'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:09.726009 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 867/1759] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:05:09 2022] DELETE /identity/v3/users/9a263de1d1074abf8756b9e07b9e9f1d => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:10.615602 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1050a-731e-4ee7-a7ef-684bee971149 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:10.616094 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1050a-731e-4ee7-a7ef-684bee971149 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:10.616166 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1050a-731e-4ee7-a7ef-684bee971149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:10.663643 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-7ea1050a-731e-4ee7-a7ef-684bee971149 None None] MFA Rules not processed for user `61f6dbd4328e459eb7d4f029551648bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:10.665056 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7ea1050a-731e-4ee7-a7ef-684bee971149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:10.816602 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f390cbf-e3fd-49b6-bab8-b5dad081c2dd admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:10.818425 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-2f390cbf-e3fd-49b6-bab8-b5dad081c2dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:10.846181 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 893/1760] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:11.004777 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f390cbf-e3fd-49b6-bab8-b5dad081c2dd admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:11.030922 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bccffe3-0322-4002-90ab-8f7e3dad472b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:11.032521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccffe3-0322-4002-90ab-8f7e3dad472b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:11.032810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccffe3-0322-4002-90ab-8f7e3dad472b admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:11.033263 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccffe3-0322-4002-90ab-8f7e3dad472b admin admin] PATH_INFO: `/v3/projects/3a2d669b5cd141408bb971741e17dd53` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:11.036561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bccffe3-0322-4002-90ab-8f7e3dad472b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a2d669b5cd141408bb971741e17dd53)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:11.038905 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bccffe3-0322-4002-90ab-8f7e3dad472b admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:11.179678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1bccffe3-0322-4002-90ab-8f7e3dad472b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a2d669b5cd141408bb971741e17dd53', 'request_id': 'req-1bccffe3-0322-4002-90ab-8f7e3dad472b'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:11.200276 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1bccffe3-0322-4002-90ab-8f7e3dad472b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a2d669b5cd141408bb971741e17dd53', 'request_id': 'req-1bccffe3-0322-4002-90ab-8f7e3dad472b'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:11.217466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1bccffe3-0322-4002-90ab-8f7e3dad472b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a2d669b5cd141408bb971741e17dd53', 'request_id': 'req-1bccffe3-0322-4002-90ab-8f7e3dad472b'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:11.249273 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1bccffe3-0322-4002-90ab-8f7e3dad472b admin admin] The token cache is being invalidate because project 3a2d669b5cd141408bb971741e17dd53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:05:11.249572 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1bccffe3-0322-4002-90ab-8f7e3dad472b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:11.253519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1bccffe3-0322-4002-90ab-8f7e3dad472b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:11.259147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 868/1761] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:05:10 2022] DELETE /identity/v3/projects/3a2d669b5cd141408bb971741e17dd53 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:11.263634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1050a-731e-4ee7-a7ef-684bee971149 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:11.297357 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1050a-731e-4ee7-a7ef-684bee971149 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:11.340398 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:11.420224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1eb26-7672-4ae7-a00e-798b9fd074e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:11.421341 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1eb26-7672-4ae7-a00e-798b9fd074e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:11.421738 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1eb26-7672-4ae7-a00e-798b9fd074e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:11.422548 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1eb26-7672-4ae7-a00e-798b9fd074e6 admin admin] PATH_INFO: `/v3/users/b5fbc2247ca24c40bdc92cf9f7c51afe` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:11.423997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e1eb26-7672-4ae7-a00e-798b9fd074e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5fbc2247ca24c40bdc92cf9f7c51afe)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:11.428440 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e1eb26-7672-4ae7-a00e-798b9fd074e6 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:11.560536 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6ca37-8956-4115-bb7d-c220ae36d1bb None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:11.561124 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6ca37-8956-4115-bb7d-c220ae36d1bb None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:11.561461 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6ca37-8956-4115-bb7d-c220ae36d1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:11.593886 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-96e1eb26-7672-4ae7-a00e-798b9fd074e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5fbc2247ca24c40bdc92cf9f7c51afe'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:11.602701 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-8cc6ca37-8956-4115-bb7d-c220ae36d1bb None None] MFA Rules not processed for user `f6b5113ea0f4484ba91a5f0e6184ca35`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:11.608673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-8cc6ca37-8956-4115-bb7d-c220ae36d1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:11.617054 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-96e1eb26-7672-4ae7-a00e-798b9fd074e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5fbc2247ca24c40bdc92cf9f7c51afe'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:11.652994 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-96e1eb26-7672-4ae7-a00e-798b9fd074e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5fbc2247ca24c40bdc92cf9f7c51afe'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:11.655354 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-96e1eb26-7672-4ae7-a00e-798b9fd074e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5fbc2247ca24c40bdc92cf9f7c51afe'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:11.658039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-96e1eb26-7672-4ae7-a00e-798b9fd074e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5fbc2247ca24c40bdc92cf9f7c51afe'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:11.701964 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 894/1762] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:05:11 2022] DELETE /identity/v3/users/b5fbc2247ca24c40bdc92cf9f7c51afe => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:11.703579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1eb26-7672-4ae7-a00e-798b9fd074e6 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:11.707525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-96e1eb26-7672-4ae7-a00e-798b9fd074e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:11.711394 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:11.785177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 869/1763] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:11.840927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1eb26-7672-4ae7-a00e-798b9fd074e6 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:11.849555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a4731-4407-4a1f-b3ed-4580e835bc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:11.850916 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a4731-4407-4a1f-b3ed-4580e835bc1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:11.851645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a4731-4407-4a1f-b3ed-4580e835bc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:11.852482 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a4731-4407-4a1f-b3ed-4580e835bc1d admin admin] PATH_INFO: `/v3/projects/1d57dcb2aae7403fb0fce84c93f5b763` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:11.854115 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a4731-4407-4a1f-b3ed-4580e835bc1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d57dcb2aae7403fb0fce84c93f5b763)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:11.856572 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a4731-4407-4a1f-b3ed-4580e835bc1d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:11.873939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce45613-85a0-4933-8906-2a8e07a84dc4 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:11.874979 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce45613-85a0-4933-8906-2a8e07a84dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:11.877192 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce45613-85a0-4933-8906-2a8e07a84dc4 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:11.879911 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 870/1764] 10.222.0.76 () {54 vars in 921 bytes} [Thu May 26 18:05:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:11.908096 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce45613-85a0-4933-8906-2a8e07a84dc4 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:11.909777 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-3ce45613-85a0-4933-8906-2a8e07a84dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:11.941142 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:11.943083 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce45613-85a0-4933-8906-2a8e07a84dc4 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:11.956384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a53a4731-4407-4a1f-b3ed-4580e835bc1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d57dcb2aae7403fb0fce84c93f5b763', 'request_id': 'req-a53a4731-4407-4a1f-b3ed-4580e835bc1d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:11.967009 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a53a4731-4407-4a1f-b3ed-4580e835bc1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d57dcb2aae7403fb0fce84c93f5b763', 'request_id': 'req-a53a4731-4407-4a1f-b3ed-4580e835bc1d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:11.977898 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a53a4731-4407-4a1f-b3ed-4580e835bc1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d57dcb2aae7403fb0fce84c93f5b763', 'request_id': 'req-a53a4731-4407-4a1f-b3ed-4580e835bc1d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:11.997705 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a53a4731-4407-4a1f-b3ed-4580e835bc1d admin admin] The token cache is being invalidate because project 1d57dcb2aae7403fb0fce84c93f5b763 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:05:11.998264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a53a4731-4407-4a1f-b3ed-4580e835bc1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:11.999146 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a53a4731-4407-4a1f-b3ed-4580e835bc1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:12.006204 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 895/1765] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:05:11 2022] DELETE /identity/v3/projects/1d57dcb2aae7403fb0fce84c93f5b763 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:12.008557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae6914-0010-42ba-a42f-c448b7ca0224 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:12.008977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae6914-0010-42ba-a42f-c448b7ca0224 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:12.009557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae6914-0010-42ba-a42f-c448b7ca0224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:12.034428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2fae6914-0010-42ba-a42f-c448b7ca0224 None None] MFA Rules not processed for user `596f126382e3440fb3cf7c357eef889f`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:12.036086 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2fae6914-0010-42ba-a42f-c448b7ca0224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:12.092341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b1fc9-7d98-4ed8-ae7f-a39394e06043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:12.093347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b1fc9-7d98-4ed8-ae7f-a39394e06043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:12.093876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b1fc9-7d98-4ed8-ae7f-a39394e06043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:12.094963 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b1fc9-7d98-4ed8-ae7f-a39394e06043 admin admin] PATH_INFO: `/v3/users/6fdb0a2dac5d4daf8a180d7c88db5332` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:12.098768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b1fc9-7d98-4ed8-ae7f-a39394e06043 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fdb0a2dac5d4daf8a180d7c88db5332)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:12.104868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b1fc9-7d98-4ed8-ae7f-a39394e06043 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:12.149454 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 896/1766] 10.222.0.76 () {58 vars in 1018 bytes} [Thu May 26 18:05:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:12.150941 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac2a7b-ec07-4b97-b13e-f5fae65c6765 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:12.151483 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac2a7b-ec07-4b97-b13e-f5fae65c6765 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:12.151797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac2a7b-ec07-4b97-b13e-f5fae65c6765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:12.207499 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c2ac2a7b-ec07-4b97-b13e-f5fae65c6765 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:12.210342 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c2ac2a7b-ec07-4b97-b13e-f5fae65c6765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:12.212743 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d89b1fc9-7d98-4ed8-ae7f-a39394e06043 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fdb0a2dac5d4daf8a180d7c88db5332'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:12.226493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d89b1fc9-7d98-4ed8-ae7f-a39394e06043 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fdb0a2dac5d4daf8a180d7c88db5332'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:12.227244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d89b1fc9-7d98-4ed8-ae7f-a39394e06043 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fdb0a2dac5d4daf8a180d7c88db5332'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:12.227869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d89b1fc9-7d98-4ed8-ae7f-a39394e06043 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fdb0a2dac5d4daf8a180d7c88db5332'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:12.231506 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 897/1767] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:12.233301 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac2a7b-ec07-4b97-b13e-f5fae65c6765 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:12.237802 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c2ac2a7b-ec07-4b97-b13e-f5fae65c6765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:12.243209 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d89b1fc9-7d98-4ed8-ae7f-a39394e06043 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fdb0a2dac5d4daf8a180d7c88db5332'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:12.271187 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac2a7b-ec07-4b97-b13e-f5fae65c6765 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:12.287619 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 871/1768] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:05:11 2022] DELETE /identity/v3/users/6fdb0a2dac5d4daf8a180d7c88db5332 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:12.290291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366187e3-d1aa-45fe-b2b5-b8f04818caa1 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:12.290810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d440ca-49b6-4d06-8cd7-50bedb9b495a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '596f126382e3440fb3cf7c357eef889f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:12.291481 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d440ca-49b6-4d06-8cd7-50bedb9b495a service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:12.291923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366187e3-d1aa-45fe-b2b5-b8f04818caa1 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:12.292209 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366187e3-d1aa-45fe-b2b5-b8f04818caa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:12.293323 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d440ca-49b6-4d06-8cd7-50bedb9b495a service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:12.293610 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d440ca-49b6-4d06-8cd7-50bedb9b495a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:12.299063 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-60d440ca-49b6-4d06-8cd7-50bedb9b495a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:12.322438 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d440ca-49b6-4d06-8cd7-50bedb9b495a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:12.326013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d440ca-49b6-4d06-8cd7-50bedb9b495a service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:12.341195 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-366187e3-d1aa-45fe-b2b5-b8f04818caa1 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:12.342406 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 898/1769] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:12.342762 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-366187e3-d1aa-45fe-b2b5-b8f04818caa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:12.483186 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:12.508647 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 872/1770] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:12.510629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d440ca-49b6-4d06-8cd7-50bedb9b495a service placement] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:12.540765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d440ca-49b6-4d06-8cd7-50bedb9b495a service placement] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:12.556757 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7cdfa-33eb-4fb0-96c4-942851b9e316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:12.557313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7cdfa-33eb-4fb0-96c4-942851b9e316 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:12.557599 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7cdfa-33eb-4fb0-96c4-942851b9e316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:12.557873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7cdfa-33eb-4fb0-96c4-942851b9e316 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:12.563918 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7cdfa-33eb-4fb0-96c4-942851b9e316 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:12.567447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7cdfa-33eb-4fb0-96c4-942851b9e316 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:12.569220 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7cdfa-33eb-4fb0-96c4-942851b9e316 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:12.611585 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 899/1771] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:12.613174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366187e3-d1aa-45fe-b2b5-b8f04818caa1 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:12.623432 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366187e3-d1aa-45fe-b2b5-b8f04818caa1 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:12.640144 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b64b64-1922-49e1-a48d-5d2a87b5622d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:12.641734 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b64b64-1922-49e1-a48d-5d2a87b5622d admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:12.641734 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b64b64-1922-49e1-a48d-5d2a87b5622d admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:12.642352 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b64b64-1922-49e1-a48d-5d2a87b5622d admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:12.642979 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b64b64-1922-49e1-a48d-5d2a87b5622d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:12.646305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b64b64-1922-49e1-a48d-5d2a87b5622d admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:12.712311 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 873/1772] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:05:12 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:12.717495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7cdfa-33eb-4fb0-96c4-942851b9e316 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:12.742931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7cdfa-33eb-4fb0-96c4-942851b9e316 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:12.942869 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca04c48-0082-40c4-b799-3fa55d5d0f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:12.943565 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca04c48-0082-40c4-b799-3fa55d5d0f03 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:12.945086 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca04c48-0082-40c4-b799-3fa55d5d0f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:12.945378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca04c48-0082-40c4-b799-3fa55d5d0f03 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:12.946153 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca04c48-0082-40c4-b799-3fa55d5d0f03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:12.950992 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca04c48-0082-40c4-b799-3fa55d5d0f03 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:12.988589 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 900/1773] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:05:12 2022] POST /identity/v3/users => generated 286 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:12.989118 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b64b64-1922-49e1-a48d-5d2a87b5622d admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:13.005779 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b64b64-1922-49e1-a48d-5d2a87b5622d admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:13.019550 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082fcb0-84cb-468a-af25-50d85f1dfc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:13.020484 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082fcb0-84cb-468a-af25-50d85f1dfc70 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:13.020897 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082fcb0-84cb-468a-af25-50d85f1dfc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:13.021637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082fcb0-84cb-468a-af25-50d85f1dfc70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:13.022539 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d082fcb0-84cb-468a-af25-50d85f1dfc70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:13.025411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d082fcb0-84cb-468a-af25-50d85f1dfc70 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:13.066100 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca04c48-0082-40c4-b799-3fa55d5d0f03 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:13.068900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 874/1774] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:13.084399 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca04c48-0082-40c4-b799-3fa55d5d0f03 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:13.094797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377108a-1c8f-44ac-9287-54539113b095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:13.097238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377108a-1c8f-44ac-9287-54539113b095 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:13.098773 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377108a-1c8f-44ac-9287-54539113b095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:13.100323 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377108a-1c8f-44ac-9287-54539113b095 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:13.100998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1377108a-1c8f-44ac-9287-54539113b095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:13.108333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1377108a-1c8f-44ac-9287-54539113b095 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:13.141920 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082fcb0-84cb-468a-af25-50d85f1dfc70 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:13.152949 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 901/1775] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:13.154147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082fcb0-84cb-468a-af25-50d85f1dfc70 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:13.158575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544bf65-088e-4161-a3bd-80cb2b840f98 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:13.158874 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544bf65-088e-4161-a3bd-80cb2b840f98 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:13.159746 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544bf65-088e-4161-a3bd-80cb2b840f98 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:13.165917 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 902/1776] 10.222.0.76 () {54 vars in 912 bytes} [Thu May 26 18:05:13 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:13.170820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e36f6db-6141-4e3a-9280-a08a2c9d93a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:13.172049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e36f6db-6141-4e3a-9280-a08a2c9d93a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:13.173259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e36f6db-6141-4e3a-9280-a08a2c9d93a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:13.173837 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e36f6db-6141-4e3a-9280-a08a2c9d93a5 admin admin] PATH_INFO: `/v3/projects/db354d763e214ca4a62b0f964ee5b60f/users/d87eaffe02374ca6b081f2d9e083af65/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:13.175867 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb58fe6-5c57-40f4-97b6-123288c0ebc8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:13.176421 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb58fe6-5c57-40f4-97b6-123288c0ebc8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:13.177543 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb58fe6-5c57-40f4-97b6-123288c0ebc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:13.216058 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:13.222540 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e36f6db-6141-4e3a-9280-a08a2c9d93a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db354d763e214ca4a62b0f964ee5b60f, user_id=d87eaffe02374ca6b081f2d9e083af65, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:13.253397 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-bdb58fe6-5c57-40f4-97b6-123288c0ebc8 None None] MFA Rules not processed for user `8df66990a3734209a350c8b6768d259f`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:13.254937 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bdb58fe6-5c57-40f4-97b6-123288c0ebc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:13.255858 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e36f6db-6141-4e3a-9280-a08a2c9d93a5 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:13.330728 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 875/1777] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:13 2022] PUT /identity/v3/projects/db354d763e214ca4a62b0f964ee5b60f/users/d87eaffe02374ca6b081f2d9e083af65/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:13.341005 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b74992-c6d8-4e12-be0f-c42e18e52f0b None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:13.341380 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b74992-c6d8-4e12-be0f-c42e18e52f0b None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:13.341679 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b74992-c6d8-4e12-be0f-c42e18e52f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:13.429847 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-02b74992-c6d8-4e12-be0f-c42e18e52f0b None None] MFA Rules not processed for user `d87eaffe02374ca6b081f2d9e083af65`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:13.431300 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-02b74992-c6d8-4e12-be0f-c42e18e52f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:13.435121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 903/1778] 10.222.0.76 () {58 vars in 1009 bytes} [Thu May 26 18:05:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:13.478792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb58fe6-5c57-40f4-97b6-123288c0ebc8 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:13.501233 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bdb58fe6-5c57-40f4-97b6-123288c0ebc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:13.556500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb58fe6-5c57-40f4-97b6-123288c0ebc8 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:13.581597 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 876/1779] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:13.582874 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b74992-c6d8-4e12-be0f-c42e18e52f0b None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:13.750063 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30050adb-9919-48a8-9648-3af81d323ae9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:13.753221 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30050adb-9919-48a8-9648-3af81d323ae9 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:13.753513 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30050adb-9919-48a8-9648-3af81d323ae9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:13.753810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30050adb-9919-48a8-9648-3af81d323ae9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:13.780851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30050adb-9919-48a8-9648-3af81d323ae9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:13.785399 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b74992-c6d8-4e12-be0f-c42e18e52f0b None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:13.786292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30050adb-9919-48a8-9648-3af81d323ae9 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:13.795599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5fcde-b363-495a-a160-e72b0b8b88be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:13.797184 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5fcde-b363-495a-a160-e72b0b8b88be admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:13.798035 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5fcde-b363-495a-a160-e72b0b8b88be admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:13.798343 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5fcde-b363-495a-a160-e72b0b8b88be admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:13.799483 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5fcde-b363-495a-a160-e72b0b8b88be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:13.801649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5fcde-b363-495a-a160-e72b0b8b88be admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:13.813964 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 904/1780] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:13.860181 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 877/1781] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:05:13 2022] POST /identity/v3/projects => generated 371 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:13.868000 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30050adb-9919-48a8-9648-3af81d323ae9 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:13.926432 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30050adb-9919-48a8-9648-3af81d323ae9 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:13.981909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5fcde-b363-495a-a160-e72b0b8b88be admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:13.985035 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-b3b5fcde-b363-495a-a160-e72b0b8b88be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:14.077614 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:14.239648 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41156e7-9023-448f-a86d-977ce2cf0abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:14.243511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41156e7-9023-448f-a86d-977ce2cf0abf admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:14.248308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41156e7-9023-448f-a86d-977ce2cf0abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:14.250115 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41156e7-9023-448f-a86d-977ce2cf0abf admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:14.252700 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41156e7-9023-448f-a86d-977ce2cf0abf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:14.275982 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41156e7-9023-448f-a86d-977ce2cf0abf admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:14.375429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 905/1782] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:05:13 2022] POST /identity/v3/users => generated 292 bytes in 509 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:14.379355 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41156e7-9023-448f-a86d-977ce2cf0abf admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:14.384688 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d41156e7-9023-448f-a86d-977ce2cf0abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:14.429477 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5fcde-b363-495a-a160-e72b0b8b88be admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:14.464395 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41156e7-9023-448f-a86d-977ce2cf0abf admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:14.490297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad39ad7-520a-4da7-a1be-7a60fc05f241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:14.494806 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fdef7c-1f9a-4b58-93fc-42cda3125862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:14.496318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad39ad7-520a-4da7-a1be-7a60fc05f241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:14.496564 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad39ad7-520a-4da7-a1be-7a60fc05f241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:14.496858 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad39ad7-520a-4da7-a1be-7a60fc05f241 admin admin] PATH_INFO: `/v3/projects/35b35e4cdd1c4e779afad8b4d0b0ed5d` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:14.498571 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad39ad7-520a-4da7-a1be-7a60fc05f241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35b35e4cdd1c4e779afad8b4d0b0ed5d)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:14.500988 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad39ad7-520a-4da7-a1be-7a60fc05f241 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:14.501713 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fdef7c-1f9a-4b58-93fc-42cda3125862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:14.505515 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fdef7c-1f9a-4b58-93fc-42cda3125862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:14.506272 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fdef7c-1f9a-4b58-93fc-42cda3125862 admin admin] PATH_INFO: `/v3/projects/642d4ba1d79e47fe83ac853019fc9539` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:14.512487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fdef7c-1f9a-4b58-93fc-42cda3125862 admin admin] RBAC: Authorizing `identity:delete_project(project_id=642d4ba1d79e47fe83ac853019fc9539)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:14.535519 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fdef7c-1f9a-4b58-93fc-42cda3125862 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:14.584473 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1ad39ad7-520a-4da7-a1be-7a60fc05f241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35b35e4cdd1c4e779afad8b4d0b0ed5d', 'request_id': 'req-1ad39ad7-520a-4da7-a1be-7a60fc05f241'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:14.600028 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1ad39ad7-520a-4da7-a1be-7a60fc05f241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35b35e4cdd1c4e779afad8b4d0b0ed5d', 'request_id': 'req-1ad39ad7-520a-4da7-a1be-7a60fc05f241'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:14.615758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1ad39ad7-520a-4da7-a1be-7a60fc05f241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35b35e4cdd1c4e779afad8b4d0b0ed5d', 'request_id': 'req-1ad39ad7-520a-4da7-a1be-7a60fc05f241'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:14.641870 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:14.643055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1ad39ad7-520a-4da7-a1be-7a60fc05f241 admin admin] The token cache is being invalidate because project 35b35e4cdd1c4e779afad8b4d0b0ed5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:05:14.644409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1ad39ad7-520a-4da7-a1be-7a60fc05f241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:14.647285 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1ad39ad7-520a-4da7-a1be-7a60fc05f241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:14.652278 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-20fdef7c-1f9a-4b58-93fc-42cda3125862 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '642d4ba1d79e47fe83ac853019fc9539', 'request_id': 'req-20fdef7c-1f9a-4b58-93fc-42cda3125862'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:14.653634 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 878/1783] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:05:13 2022] DELETE /identity/v3/projects/35b35e4cdd1c4e779afad8b4d0b0ed5d => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:14.655853 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad39ad7-520a-4da7-a1be-7a60fc05f241 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:14.659119 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-1ad39ad7-520a-4da7-a1be-7a60fc05f241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:14.668632 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-20fdef7c-1f9a-4b58-93fc-42cda3125862 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '642d4ba1d79e47fe83ac853019fc9539', 'request_id': 'req-20fdef7c-1f9a-4b58-93fc-42cda3125862'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:14.684502 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-20fdef7c-1f9a-4b58-93fc-42cda3125862 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '642d4ba1d79e47fe83ac853019fc9539', 'request_id': 'req-20fdef7c-1f9a-4b58-93fc-42cda3125862'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:14.715046 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-20fdef7c-1f9a-4b58-93fc-42cda3125862 admin admin] The token cache is being invalidate because project 642d4ba1d79e47fe83ac853019fc9539 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:05:14.715384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-20fdef7c-1f9a-4b58-93fc-42cda3125862 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:14.717966 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-20fdef7c-1f9a-4b58-93fc-42cda3125862 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:14.721820 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 906/1784] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:05:14 2022] DELETE /identity/v3/projects/642d4ba1d79e47fe83ac853019fc9539 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:14.730419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3625a73-5405-4d6f-9c5b-4e21a427ee5f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:14.731072 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3625a73-5405-4d6f-9c5b-4e21a427ee5f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:14.731530 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3625a73-5405-4d6f-9c5b-4e21a427ee5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:14.777720 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f3625a73-5405-4d6f-9c5b-4e21a427ee5f None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:14.780865 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f3625a73-5405-4d6f-9c5b-4e21a427ee5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:14.879935 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad39ad7-520a-4da7-a1be-7a60fc05f241 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:14.994475 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 907/1785] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:15.011578 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3625a73-5405-4d6f-9c5b-4e21a427ee5f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:15.021228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f3625a73-5405-4d6f-9c5b-4e21a427ee5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:15.068221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:15.071287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3625a73-5405-4d6f-9c5b-4e21a427ee5f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:15.095087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49637161-e8cf-4c21-85fa-6e4a11ad35f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:15.095785 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49637161-e8cf-4c21-85fa-6e4a11ad35f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:15.096197 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49637161-e8cf-4c21-85fa-6e4a11ad35f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:15.096473 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49637161-e8cf-4c21-85fa-6e4a11ad35f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:15.097489 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49637161-e8cf-4c21-85fa-6e4a11ad35f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:15.099133 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49637161-e8cf-4c21-85fa-6e4a11ad35f3 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:15.104618 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972e8c7-8d8d-44ba-aa15-9725741d8824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:15.107518 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972e8c7-8d8d-44ba-aa15-9725741d8824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:15.108034 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972e8c7-8d8d-44ba-aa15-9725741d8824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:15.108551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972e8c7-8d8d-44ba-aa15-9725741d8824 admin admin] PATH_INFO: `/v3/users/e5e66ed4a9b346a68bb70206027946a9` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:15.117195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972e8c7-8d8d-44ba-aa15-9725741d8824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5e66ed4a9b346a68bb70206027946a9)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:15.127691 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 879/1786] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 473 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:15.130134 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30090d1-4a76-4372-bcb9-ef226424657a None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:15.130134 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30090d1-4a76-4372-bcb9-ef226424657a None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:15.130362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30090d1-4a76-4372-bcb9-ef226424657a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:15.134308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972e8c7-8d8d-44ba-aa15-9725741d8824 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:15.223812 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-a30090d1-4a76-4372-bcb9-ef226424657a None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:15.229143 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-a30090d1-4a76-4372-bcb9-ef226424657a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:15.266878 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 880/1787] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:15.269484 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30090d1-4a76-4372-bcb9-ef226424657a None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:15.280786 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-a30090d1-4a76-4372-bcb9-ef226424657a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:15.296959 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d972e8c7-8d8d-44ba-aa15-9725741d8824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5e66ed4a9b346a68bb70206027946a9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:15.313502 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30090d1-4a76-4372-bcb9-ef226424657a None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:15.331789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ede28-7aee-48a5-9e0d-c81625be0fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:15.343717 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ede28-7aee-48a5-9e0d-c81625be0fa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:15.344505 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ede28-7aee-48a5-9e0d-c81625be0fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:15.344932 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ede28-7aee-48a5-9e0d-c81625be0fa0 admin admin] PATH_INFO: `/v3/projects/8322d85b7ebd468ca4ee85d9d94e19be/users/e5585e732f93478e87be9f8f8815efe4/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:15.345307 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d972e8c7-8d8d-44ba-aa15-9725741d8824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5e66ed4a9b346a68bb70206027946a9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:15.404747 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ede28-7aee-48a5-9e0d-c81625be0fa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8322d85b7ebd468ca4ee85d9d94e19be, user_id=e5585e732f93478e87be9f8f8815efe4, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:15.406462 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d972e8c7-8d8d-44ba-aa15-9725741d8824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5e66ed4a9b346a68bb70206027946a9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:15.409528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d972e8c7-8d8d-44ba-aa15-9725741d8824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5e66ed4a9b346a68bb70206027946a9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:15.413390 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d972e8c7-8d8d-44ba-aa15-9725741d8824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5e66ed4a9b346a68bb70206027946a9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:15.415523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ede28-7aee-48a5-9e0d-c81625be0fa0 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:15.471796 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 908/1788] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:05:15 2022] DELETE /identity/v3/users/e5e66ed4a9b346a68bb70206027946a9 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:15.476707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972e8c7-8d8d-44ba-aa15-9725741d8824 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:15.476707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d972e8c7-8d8d-44ba-aa15-9725741d8824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:15.482256 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 881/1789] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:15 2022] PUT /identity/v3/projects/8322d85b7ebd468ca4ee85d9d94e19be/users/e5585e732f93478e87be9f8f8815efe4/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:15.489382 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ede28-7aee-48a5-9e0d-c81625be0fa0 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:15.496789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-dd0ede28-7aee-48a5-9e0d-c81625be0fa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:15.648229 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:15.738083 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972e8c7-8d8d-44ba-aa15-9725741d8824 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:15.753550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b3b5e0-1ba4-4fd0-a59a-62779ccba35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:15.754403 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3b5e0-1ba4-4fd0-a59a-62779ccba35a admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:15.754855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3b5e0-1ba4-4fd0-a59a-62779ccba35a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:15.755132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3b5e0-1ba4-4fd0-a59a-62779ccba35a admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:15.756042 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b3b5e0-1ba4-4fd0-a59a-62779ccba35a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:15.756402 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b3b5e0-1ba4-4fd0-a59a-62779ccba35a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:15.759329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b3b5e0-1ba4-4fd0-a59a-62779ccba35a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:15.786986 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ede28-7aee-48a5-9e0d-c81625be0fa0 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:15.787420 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bf432-edcc-4bf8-9612-b0fc6a3da15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:15.787911 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bf432-edcc-4bf8-9612-b0fc6a3da15d admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:15.788197 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bf432-edcc-4bf8-9612-b0fc6a3da15d admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:15.788476 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bf432-edcc-4bf8-9612-b0fc6a3da15d admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:15.789216 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bf432-edcc-4bf8-9612-b0fc6a3da15d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:15.790892 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bf432-edcc-4bf8-9612-b0fc6a3da15d admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:15.812234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 882/1790] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:15.812234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7163d2-0068-4f72-8064-a971a07e0c72 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:15.812234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7163d2-0068-4f72-8064-a971a07e0c72 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:15.812234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7163d2-0068-4f72-8064-a971a07e0c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:15.820685 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 909/1791] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:15.821549 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b3b5e0-1ba4-4fd0-a59a-62779ccba35a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:15.844540 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-1b7163d2-0068-4f72-8064-a971a07e0c72 None None] MFA Rules not processed for user `dfbe23f3a0294148a88f4758d8470665`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:15.847014 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-1b7163d2-0068-4f72-8064-a971a07e0c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:15.870879 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b3b5e0-1ba4-4fd0-a59a-62779ccba35a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:15.889239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525ca75-9540-42ea-86cf-c198129f25cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:15.890446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525ca75-9540-42ea-86cf-c198129f25cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:15.892515 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525ca75-9540-42ea-86cf-c198129f25cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:15.893040 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525ca75-9540-42ea-86cf-c198129f25cf admin admin] PATH_INFO: `/v3/projects/8322d85b7ebd468ca4ee85d9d94e19be/users/e5585e732f93478e87be9f8f8815efe4/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:15.900191 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525ca75-9540-42ea-86cf-c198129f25cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8322d85b7ebd468ca4ee85d9d94e19be, user_id=e5585e732f93478e87be9f8f8815efe4, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:15.909396 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525ca75-9540-42ea-86cf-c198129f25cf admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:15.944880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 910/1792] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:15 2022] PUT /identity/v3/projects/8322d85b7ebd468ca4ee85d9d94e19be/users/e5585e732f93478e87be9f8f8815efe4/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:15.954171 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525ca75-9540-42ea-86cf-c198129f25cf admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:15.980726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 883/1793] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:15.980726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a191465-a208-4872-886c-5a2adb255dd9 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:15.980726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a191465-a208-4872-886c-5a2adb255dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:15.981302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a191465-a208-4872-886c-5a2adb255dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:16.057606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-2a191465-a208-4872-886c-5a2adb255dd9 None None] MFA Rules not processed for user `e5585e732f93478e87be9f8f8815efe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:16.058891 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-2a191465-a208-4872-886c-5a2adb255dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:16.207145 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:16.208016 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525ca75-9540-42ea-86cf-c198129f25cf admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:16.222841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb41c4f-cc38-4ab2-9260-f34c910d0451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:16.223651 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb41c4f-cc38-4ab2-9260-f34c910d0451 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:16.223937 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb41c4f-cc38-4ab2-9260-f34c910d0451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:16.224224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb41c4f-cc38-4ab2-9260-f34c910d0451 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:16.227519 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb41c4f-cc38-4ab2-9260-f34c910d0451 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:16.230523 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb41c4f-cc38-4ab2-9260-f34c910d0451 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:16.255141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 884/1794] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:15 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:16.256105 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a191465-a208-4872-886c-5a2adb255dd9 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:16.261648 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-2a191465-a208-4872-886c-5a2adb255dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:16.303351 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a191465-a208-4872-886c-5a2adb255dd9 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:16.316612 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a066ae4-095c-43fb-9ce0-3757362451ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:16.317887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a066ae4-095c-43fb-9ce0-3757362451ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:16.319392 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a066ae4-095c-43fb-9ce0-3757362451ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:16.320446 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a066ae4-095c-43fb-9ce0-3757362451ec admin admin] PATH_INFO: `/v3/projects/3758efd2fdea4a34b0fae1496bb7c14a` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:16.325352 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a066ae4-095c-43fb-9ce0-3757362451ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=3758efd2fdea4a34b0fae1496bb7c14a)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:16.331540 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a066ae4-095c-43fb-9ce0-3757362451ec admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:16.339263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 911/1795] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:05:15 2022] POST /identity/v3/projects => generated 383 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:16.339263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb41c4f-cc38-4ab2-9260-f34c910d0451 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:16.346768 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2cb41c4f-cc38-4ab2-9260-f34c910d0451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:16.492029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1a066ae4-095c-43fb-9ce0-3757362451ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3758efd2fdea4a34b0fae1496bb7c14a', 'request_id': 'req-1a066ae4-095c-43fb-9ce0-3757362451ec'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:16.509406 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1a066ae4-095c-43fb-9ce0-3757362451ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3758efd2fdea4a34b0fae1496bb7c14a', 'request_id': 'req-1a066ae4-095c-43fb-9ce0-3757362451ec'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:16.525292 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1a066ae4-095c-43fb-9ce0-3757362451ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3758efd2fdea4a34b0fae1496bb7c14a', 'request_id': 'req-1a066ae4-095c-43fb-9ce0-3757362451ec'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:16.563566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1a066ae4-095c-43fb-9ce0-3757362451ec admin admin] The token cache is being invalidate because project 3758efd2fdea4a34b0fae1496bb7c14a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:05:16.564094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1a066ae4-095c-43fb-9ce0-3757362451ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:16.566243 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1a066ae4-095c-43fb-9ce0-3757362451ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:16.572016 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb41c4f-cc38-4ab2-9260-f34c910d0451 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:16.576117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 885/1796] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:05:16 2022] DELETE /identity/v3/projects/3758efd2fdea4a34b0fae1496bb7c14a => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:16.580694 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a066ae4-095c-43fb-9ce0-3757362451ec admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:16.583066 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-1a066ae4-095c-43fb-9ce0-3757362451ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:16.760182 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905cef3-2c6e-41a6-9ac0-565b67f68b99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:16.765449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905cef3-2c6e-41a6-9ac0-565b67f68b99 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:16.767607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905cef3-2c6e-41a6-9ac0-565b67f68b99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:16.770404 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905cef3-2c6e-41a6-9ac0-565b67f68b99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:16.775134 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2905cef3-2c6e-41a6-9ac0-565b67f68b99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:16.777211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a066ae4-095c-43fb-9ce0-3757362451ec admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:16.790281 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c834ceb-4c87-4ab7-b991-a11c9e45681a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:16.790994 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c834ceb-4c87-4ab7-b991-a11c9e45681a admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:16.791281 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c834ceb-4c87-4ab7-b991-a11c9e45681a admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:16.791562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c834ceb-4c87-4ab7-b991-a11c9e45681a admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:16.795185 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c834ceb-4c87-4ab7-b991-a11c9e45681a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:16.797596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c834ceb-4c87-4ab7-b991-a11c9e45681a admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:16.850262 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 886/1797] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:05:16 2022] POST /identity/v3/users => generated 292 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:16.851896 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79608538-4b33-432e-a492-1694ebf16a47 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:16.852294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79608538-4b33-432e-a492-1694ebf16a47 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:16.852577 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79608538-4b33-432e-a492-1694ebf16a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:16.856021 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:16.904997 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-79608538-4b33-432e-a492-1694ebf16a47 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:16.907785 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-79608538-4b33-432e-a492-1694ebf16a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:16.933205 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 887/1798] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:16.944327 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79608538-4b33-432e-a492-1694ebf16a47 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:16.958976 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79608538-4b33-432e-a492-1694ebf16a47 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:16.972973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad1385-517b-4af9-9bab-13b09abccd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:16.973676 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad1385-517b-4af9-9bab-13b09abccd46 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:16.973956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad1385-517b-4af9-9bab-13b09abccd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:16.974251 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad1385-517b-4af9-9bab-13b09abccd46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:16.976102 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad1385-517b-4af9-9bab-13b09abccd46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:16.979360 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad1385-517b-4af9-9bab-13b09abccd46 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:16.995292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905cef3-2c6e-41a6-9ac0-565b67f68b99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:16.997732 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905cef3-2c6e-41a6-9ac0-565b67f68b99 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:17.008278 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 888/1799] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:17.009144 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad1385-517b-4af9-9bab-13b09abccd46 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:17.012529 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-d5ad1385-517b-4af9-9bab-13b09abccd46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:17.025986 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 912/1800] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:16 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:17.029634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6470ba-492b-46bb-a980-c11e8df28116 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:17.030078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6470ba-492b-46bb-a980-c11e8df28116 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:17.030665 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6470ba-492b-46bb-a980-c11e8df28116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:17.038279 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad1385-517b-4af9-9bab-13b09abccd46 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:17.056335 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b525a5d-0f90-4096-9992-bb9130da2c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:17.057337 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b525a5d-0f90-4096-9992-bb9130da2c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:17.057720 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b525a5d-0f90-4096-9992-bb9130da2c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:17.058095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b525a5d-0f90-4096-9992-bb9130da2c45 admin admin] PATH_INFO: `/v3/users/54ffdd4c8cfd41319a04db3cebdadf84` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:17.059550 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b525a5d-0f90-4096-9992-bb9130da2c45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54ffdd4c8cfd41319a04db3cebdadf84)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:17.062319 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b525a5d-0f90-4096-9992-bb9130da2c45 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:17.073557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-9c6470ba-492b-46bb-a980-c11e8df28116 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:17.077109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9c6470ba-492b-46bb-a980-c11e8df28116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:17.112749 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:17.121145 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 913/1801] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:17.122632 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6470ba-492b-46bb-a980-c11e8df28116 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:17.154840 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6470ba-492b-46bb-a980-c11e8df28116 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:17.185291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-7b525a5d-0f90-4096-9992-bb9130da2c45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54ffdd4c8cfd41319a04db3cebdadf84'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:17.193893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e6b73-8bcf-4bcf-beee-6f7733608090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:17.194753 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e6b73-8bcf-4bcf-beee-6f7733608090 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:17.196179 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e6b73-8bcf-4bcf-beee-6f7733608090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:17.196573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e6b73-8bcf-4bcf-beee-6f7733608090 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:17.197666 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832e6b73-8bcf-4bcf-beee-6f7733608090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:17.200773 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832e6b73-8bcf-4bcf-beee-6f7733608090 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:17.204575 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-7b525a5d-0f90-4096-9992-bb9130da2c45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54ffdd4c8cfd41319a04db3cebdadf84'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:17.205015 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-7b525a5d-0f90-4096-9992-bb9130da2c45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54ffdd4c8cfd41319a04db3cebdadf84'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:17.205757 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-7b525a5d-0f90-4096-9992-bb9130da2c45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54ffdd4c8cfd41319a04db3cebdadf84'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:17.242446 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-7b525a5d-0f90-4096-9992-bb9130da2c45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54ffdd4c8cfd41319a04db3cebdadf84'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:17.258716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 914/1802] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:17.260748 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e6b73-8bcf-4bcf-beee-6f7733608090 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:17.269242 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-832e6b73-8bcf-4bcf-beee-6f7733608090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:17.307030 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 889/1803] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:05:17 2022] DELETE /identity/v3/users/54ffdd4c8cfd41319a04db3cebdadf84 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:17.308303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b525a5d-0f90-4096-9992-bb9130da2c45 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:17.308896 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e6b73-8bcf-4bcf-beee-6f7733608090 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:17.310529 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-7b525a5d-0f90-4096-9992-bb9130da2c45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:17.386039 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b525a5d-0f90-4096-9992-bb9130da2c45 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:17.395665 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a17b4e9-1ec9-45e4-af38-e290c15ada90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:17.396541 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17b4e9-1ec9-45e4-af38-e290c15ada90 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:17.396920 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17b4e9-1ec9-45e4-af38-e290c15ada90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:17.397295 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17b4e9-1ec9-45e4-af38-e290c15ada90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:17.399232 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26e481-76df-4bdf-93b9-3bfaf4497c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:17.399379 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-7a17b4e9-1ec9-45e4-af38-e290c15ada90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:17.400043 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26e481-76df-4bdf-93b9-3bfaf4497c9a admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:17.400620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26e481-76df-4bdf-93b9-3bfaf4497c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:17.401058 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26e481-76df-4bdf-93b9-3bfaf4497c9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:17.401906 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e26e481-76df-4bdf-93b9-3bfaf4497c9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:17.402446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e26e481-76df-4bdf-93b9-3bfaf4497c9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:17.404683 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e26e481-76df-4bdf-93b9-3bfaf4497c9a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:17.424403 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 915/1804] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:17.425986 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26e481-76df-4bdf-93b9-3bfaf4497c9a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:17.430029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0e26e481-76df-4bdf-93b9-3bfaf4497c9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:17.450963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26e481-76df-4bdf-93b9-3bfaf4497c9a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:17.461553 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34b18d-2985-44d3-a659-bdf15d3ded93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:17.462581 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34b18d-2985-44d3-a659-bdf15d3ded93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:17.463214 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34b18d-2985-44d3-a659-bdf15d3ded93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:17.463214 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34b18d-2985-44d3-a659-bdf15d3ded93 admin admin] PATH_INFO: `/v3/projects/eae794b2af6b47cc8b3efd9142467f2f/users/650b4f20681444c5b85ba22c9424b05b/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:17.488554 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34b18d-2985-44d3-a659-bdf15d3ded93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae794b2af6b47cc8b3efd9142467f2f, user_id=650b4f20681444c5b85ba22c9424b05b, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:17.492420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34b18d-2985-44d3-a659-bdf15d3ded93 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:17.510550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 916/1805] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:17 2022] PUT /identity/v3/projects/eae794b2af6b47cc8b3efd9142467f2f/users/650b4f20681444c5b85ba22c9424b05b/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:17.512275 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396e499-4f9b-414f-b64f-e201a7f4587d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:17.512574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396e499-4f9b-414f-b64f-e201a7f4587d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:17.512858 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396e499-4f9b-414f-b64f-e201a7f4587d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:17.530809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a17b4e9-1ec9-45e4-af38-e290c15ada90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:17.534344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a17b4e9-1ec9-45e4-af38-e290c15ada90 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:17.556146 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 890/1806] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:17.557442 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a17b4e9-1ec9-45e4-af38-e290c15ada90 service cinder] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:17.569091 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6396e499-4f9b-414f-b64f-e201a7f4587d None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:17.570567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6396e499-4f9b-414f-b64f-e201a7f4587d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:17.585826 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a17b4e9-1ec9-45e4-af38-e290c15ada90 service cinder] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:17.777050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67acd54-f587-4c84-ac6d-c996f66dc08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:17.781550 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67acd54-f587-4c84-ac6d-c996f66dc08c admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:17.796513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67acd54-f587-4c84-ac6d-c996f66dc08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:17.800969 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67acd54-f587-4c84-ac6d-c996f66dc08c admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:17.801699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 917/1807] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:17.803357 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814b2ff-55d5-43a7-a58d-cd49765734fd None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:17.803579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814b2ff-55d5-43a7-a58d-cd49765734fd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:17.803821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814b2ff-55d5-43a7-a58d-cd49765734fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:17.815789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67acd54-f587-4c84-ac6d-c996f66dc08c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:17.829627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67acd54-f587-4c84-ac6d-c996f66dc08c admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:17.847787 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:17.869214 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4814b2ff-55d5-43a7-a58d-cd49765734fd None None] MFA Rules not processed for user `650b4f20681444c5b85ba22c9424b05b`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:17.871074 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4814b2ff-55d5-43a7-a58d-cd49765734fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:17.882893 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:17.931041 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 891/1808] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:05:17 2022] POST /identity/v3/projects => generated 357 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:17.932923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67acd54-f587-4c84-ac6d-c996f66dc08c admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:17.936186 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-a67acd54-f587-4c84-ac6d-c996f66dc08c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:18.113466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 918/1809] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:18.113466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814b2ff-55d5-43a7-a58d-cd49765734fd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:18.151231 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814b2ff-55d5-43a7-a58d-cd49765734fd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:18.190881 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67acd54-f587-4c84-ac6d-c996f66dc08c admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:18.216138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0b575-5646-4235-aedc-1d67256c466c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:18.221258 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0b575-5646-4235-aedc-1d67256c466c service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:18.228176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0b575-5646-4235-aedc-1d67256c466c service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:18.228688 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0b575-5646-4235-aedc-1d67256c466c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:18.343452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16c186-d846-4e9a-89e4-8195a98ba749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:18.345090 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16c186-d846-4e9a-89e4-8195a98ba749 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:18.345090 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16c186-d846-4e9a-89e4-8195a98ba749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:18.345090 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16c186-d846-4e9a-89e4-8195a98ba749 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:18.345528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea16c186-d846-4e9a-89e4-8195a98ba749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:18.350550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea16c186-d846-4e9a-89e4-8195a98ba749 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:18.377522 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff0b575-5646-4235-aedc-1d67256c466c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:18.383984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff0b575-5646-4235-aedc-1d67256c466c service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:18.396009 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 919/1810] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:05:18 2022] POST /identity/v3/users => generated 279 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:18.396009 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16c186-d846-4e9a-89e4-8195a98ba749 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:18.415515 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16c186-d846-4e9a-89e4-8195a98ba749 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:18.424264 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 892/1811] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:18.428632 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0b575-5646-4235-aedc-1d67256c466c service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:18.431899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb75a0-be2c-432d-b17d-ee0e93e51c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:18.432595 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb75a0-be2c-432d-b17d-ee0e93e51c75 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:18.432886 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb75a0-be2c-432d-b17d-ee0e93e51c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:18.433311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb75a0-be2c-432d-b17d-ee0e93e51c75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:18.434357 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb75a0-be2c-432d-b17d-ee0e93e51c75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:18.436165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb75a0-be2c-432d-b17d-ee0e93e51c75 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:18.462617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0b575-5646-4235-aedc-1d67256c466c service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:18.483490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0ac2a-8acd-4814-8e8d-e8351e7d3185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:18.487246 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0ac2a-8acd-4814-8e8d-e8351e7d3185 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:18.488012 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0ac2a-8acd-4814-8e8d-e8351e7d3185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:18.488561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0ac2a-8acd-4814-8e8d-e8351e7d3185 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:18.490228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0ac2a-8acd-4814-8e8d-e8351e7d3185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:18.491195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 920/1812] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:05:18 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:18.492758 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758ebf5-ced6-4105-9ea2-c7522339dfca None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:18.493062 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758ebf5-ced6-4105-9ea2-c7522339dfca None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:18.493347 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758ebf5-ced6-4105-9ea2-c7522339dfca None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:18.494696 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 921/1813] 10.222.0.76 () {54 vars in 921 bytes} [Thu May 26 18:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:18.495020 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0ac2a-8acd-4814-8e8d-e8351e7d3185 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:18.499146 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758ebf5-ced6-4105-9ea2-c7522339dfca None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:18.515619 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 893/1814] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:18.519629 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf87d31-fd4c-493e-81a3-425ebc44b152 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:18.519984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf87d31-fd4c-493e-81a3-425ebc44b152 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:18.520404 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf87d31-fd4c-493e-81a3-425ebc44b152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:18.549276 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-edf87d31-fd4c-493e-81a3-425ebc44b152 None None] MFA Rules not processed for user `1899c317083d40f5b490cd9eb63fe521`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:18.551582 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-edf87d31-fd4c-493e-81a3-425ebc44b152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:18.702032 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 894/1815] 10.222.0.76 () {58 vars in 1018 bytes} [Thu May 26 18:05:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:18.705592 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf87d31-fd4c-493e-81a3-425ebc44b152 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:18.714114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758ebf5-ced6-4105-9ea2-c7522339dfca None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:18.727287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246be79-23a1-4ff8-adb8-f6364a39892c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:18.727988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246be79-23a1-4ff8-adb8-f6364a39892c admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:18.728272 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246be79-23a1-4ff8-adb8-f6364a39892c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:18.728549 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246be79-23a1-4ff8-adb8-f6364a39892c admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:18.729253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246be79-23a1-4ff8-adb8-f6364a39892c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:18.730643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf87d31-fd4c-493e-81a3-425ebc44b152 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:18.734737 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246be79-23a1-4ff8-adb8-f6364a39892c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:18.746228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a5156-cf6c-4183-a444-684cae21f714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:18.747000 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a5156-cf6c-4183-a444-684cae21f714 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:18.747288 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a5156-cf6c-4183-a444-684cae21f714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:18.747563 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a5156-cf6c-4183-a444-684cae21f714 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:18.748295 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a5156-cf6c-4183-a444-684cae21f714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:18.749900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a5156-cf6c-4183-a444-684cae21f714 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:18.758100 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:18.779431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 922/1816] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:05:18 2022] POST /identity/v3/users => generated 298 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:18.784436 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246be79-23a1-4ff8-adb8-f6364a39892c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:18.785809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 895/1817] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:18.788486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a5156-cf6c-4183-a444-684cae21f714 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:18.789473 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8246be79-23a1-4ff8-adb8-f6364a39892c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:18.813904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a5156-cf6c-4183-a444-684cae21f714 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:18.825978 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922df8d-5aac-4987-9928-6da8b16f21fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:18.828253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922df8d-5aac-4987-9928-6da8b16f21fc admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:18.829695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922df8d-5aac-4987-9928-6da8b16f21fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:18.829990 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922df8d-5aac-4987-9928-6da8b16f21fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:18.832700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7922df8d-5aac-4987-9928-6da8b16f21fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:18.833618 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246be79-23a1-4ff8-adb8-f6364a39892c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:18.836328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7922df8d-5aac-4987-9928-6da8b16f21fc admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:18.847219 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:18.854590 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d2ed9-369d-4452-bbca-4f9af83fdbdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:18.855420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d2ed9-369d-4452-bbca-4f9af83fdbdb service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:18.855718 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d2ed9-369d-4452-bbca-4f9af83fdbdb service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:18.856073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d2ed9-369d-4452-bbca-4f9af83fdbdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:18.862912 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 896/1818] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:18.862912 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922df8d-5aac-4987-9928-6da8b16f21fc admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:18.880309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d2ed9-369d-4452-bbca-4f9af83fdbdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:18.887175 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922df8d-5aac-4987-9928-6da8b16f21fc admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:18.888180 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d2ed9-369d-4452-bbca-4f9af83fdbdb service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:18.895651 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871f97b-515d-42da-a472-10f32a3930c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:18.897310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871f97b-515d-42da-a472-10f32a3930c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:18.897709 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871f97b-515d-42da-a472-10f32a3930c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:18.898071 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871f97b-515d-42da-a472-10f32a3930c7 admin admin] PATH_INFO: `/v3/projects/dbb8c80a56f544c9a0203db42382f72f/users/8b9579d5fe3f4bb5a7fa432013f304e6/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:18.920605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871f97b-515d-42da-a472-10f32a3930c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbb8c80a56f544c9a0203db42382f72f, user_id=8b9579d5fe3f4bb5a7fa432013f304e6, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:18.924341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871f97b-515d-42da-a472-10f32a3930c7 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:18.925898 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 923/1819] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:18.927330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d2ed9-369d-4452-bbca-4f9af83fdbdb service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:18.964617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d2ed9-369d-4452-bbca-4f9af83fdbdb service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:18.983819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 897/1820] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:18 2022] PUT /identity/v3/projects/dbb8c80a56f544c9a0203db42382f72f/users/8b9579d5fe3f4bb5a7fa432013f304e6/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:18.985888 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e1bf4-bd64-4256-bf24-2ef24898616a None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:18.986312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e1bf4-bd64-4256-bf24-2ef24898616a None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:18.986689 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e1bf4-bd64-4256-bf24-2ef24898616a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:19.061452 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-615e1bf4-bd64-4256-bf24-2ef24898616a None None] MFA Rules not processed for user `8b9579d5fe3f4bb5a7fa432013f304e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:19.063428 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-615e1bf4-bd64-4256-bf24-2ef24898616a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:19.118846 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307d668-29b2-4596-98ee-f1db9499cf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:19.119980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307d668-29b2-4596-98ee-f1db9499cf91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:19.120468 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307d668-29b2-4596-98ee-f1db9499cf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:19.122565 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307d668-29b2-4596-98ee-f1db9499cf91 admin admin] PATH_INFO: `/v3/projects/50ed3c4298544707945fda2d53e8112a/users/306e8ec162d04074906fd9ec1b9e936e/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:19.164318 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307d668-29b2-4596-98ee-f1db9499cf91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50ed3c4298544707945fda2d53e8112a, user_id=306e8ec162d04074906fd9ec1b9e936e, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:19.185899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307d668-29b2-4596-98ee-f1db9499cf91 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:19.219683 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 898/1821] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:18 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:05:19.220141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 924/1822] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:18 2022] PUT /identity/v3/projects/50ed3c4298544707945fda2d53e8112a/users/306e8ec162d04074906fd9ec1b9e936e/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:19.220912 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e1bf4-bd64-4256-bf24-2ef24898616a None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:19.224428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307d668-29b2-4596-98ee-f1db9499cf91 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:19.240102 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e1bf4-bd64-4256-bf24-2ef24898616a None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:19.337864 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307d668-29b2-4596-98ee-f1db9499cf91 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:19.370076 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7f75a-773c-4440-a141-c6eb55078fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:19.371166 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7f75a-773c-4440-a141-c6eb55078fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:19.372895 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6c45e-2814-4721-bafa-20dec89e9f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:19.374689 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7f75a-773c-4440-a141-c6eb55078fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:19.375849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7f75a-773c-4440-a141-c6eb55078fc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:19.376718 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6c45e-2814-4721-bafa-20dec89e9f9e admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:19.377048 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6c45e-2814-4721-bafa-20dec89e9f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:19.378877 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7f75a-773c-4440-a141-c6eb55078fc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:19.379254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6c45e-2814-4721-bafa-20dec89e9f9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:19.380092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6c45e-2814-4721-bafa-20dec89e9f9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:19.388064 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7f75a-773c-4440-a141-c6eb55078fc0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:19.390471 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6c45e-2814-4721-bafa-20dec89e9f9e admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:19.419900 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 925/1823] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:19.425570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7f75a-773c-4440-a141-c6eb55078fc0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:19.450901 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7f75a-773c-4440-a141-c6eb55078fc0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:19.460648 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 899/1824] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:05:19 2022] POST /identity/v3/projects => generated 357 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:05:19.462775 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6c45e-2814-4721-bafa-20dec89e9f9e admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:19.475093 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f908406-8b4c-4cb6-b5c0-e009aeb4c0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:19.477501 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f908406-8b4c-4cb6-b5c0-e009aeb4c0d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:19.479792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f908406-8b4c-4cb6-b5c0-e009aeb4c0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:19.480174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f908406-8b4c-4cb6-b5c0-e009aeb4c0d0 admin admin] PATH_INFO: `/v3/projects/50ed3c4298544707945fda2d53e8112a/users/306e8ec162d04074906fd9ec1b9e936e/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:19.491527 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f908406-8b4c-4cb6-b5c0-e009aeb4c0d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50ed3c4298544707945fda2d53e8112a, user_id=306e8ec162d04074906fd9ec1b9e936e, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:19.498520 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f908406-8b4c-4cb6-b5c0-e009aeb4c0d0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:19.553491 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 926/1825] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:19 2022] PUT /identity/v3/projects/50ed3c4298544707945fda2d53e8112a/users/306e8ec162d04074906fd9ec1b9e936e/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:19.556751 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331c1aa-343a-4d0d-bf36-d3dc082f0c24 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:19.567210 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331c1aa-343a-4d0d-bf36-d3dc082f0c24 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:19.567568 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331c1aa-343a-4d0d-bf36-d3dc082f0c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:19.678436 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4331c1aa-343a-4d0d-bf36-d3dc082f0c24 None None] MFA Rules not processed for user `306e8ec162d04074906fd9ec1b9e936e`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:19.678436 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4331c1aa-343a-4d0d-bf36-d3dc082f0c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:19.686300 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:19.715696 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6c45e-2814-4721-bafa-20dec89e9f9e admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:19.795442 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:19.884386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 927/1826] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:05:19.885438 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331c1aa-343a-4d0d-bf36-d3dc082f0c24 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:19.898659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4331c1aa-343a-4d0d-bf36-d3dc082f0c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:19.913839 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d0e58-86f6-4216-86ef-bcb4bac514e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:19.914772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d0e58-86f6-4216-86ef-bcb4bac514e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:19.915088 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d0e58-86f6-4216-86ef-bcb4bac514e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:19.916469 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d0e58-86f6-4216-86ef-bcb4bac514e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:19.918119 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d0e58-86f6-4216-86ef-bcb4bac514e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:19.923930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d0e58-86f6-4216-86ef-bcb4bac514e0 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:19.972293 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 900/1827] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:05:19 2022] POST /identity/v3/users => generated 285 bytes in 511 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:19.973425 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d0e58-86f6-4216-86ef-bcb4bac514e0 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:19.975718 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-265d0e58-86f6-4216-86ef-bcb4bac514e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:20.026337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331c1aa-343a-4d0d-bf36-d3dc082f0c24 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:20.034230 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fee573-dc77-40e2-a184-5f0df28a652c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:20.035237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fee573-dc77-40e2-a184-5f0df28a652c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:20.035525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fee573-dc77-40e2-a184-5f0df28a652c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:20.035800 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fee573-dc77-40e2-a184-5f0df28a652c admin admin] PATH_INFO: `/v3/projects/204f8f384bc54e699f26d5f9ba6bc590` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:20.037114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fee573-dc77-40e2-a184-5f0df28a652c admin admin] RBAC: Authorizing `identity:delete_project(project_id=204f8f384bc54e699f26d5f9ba6bc590)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:20.040311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fee573-dc77-40e2-a184-5f0df28a652c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:20.051684 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d0e58-86f6-4216-86ef-bcb4bac514e0 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:20.055938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8c3a0-7e0f-4ff9-bcee-b02b1a0e0be0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:20.056654 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8c3a0-7e0f-4ff9-bcee-b02b1a0e0be0 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:20.056939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8c3a0-7e0f-4ff9-bcee-b02b1a0e0be0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:20.057217 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8c3a0-7e0f-4ff9-bcee-b02b1a0e0be0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:20.073896 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e8c3a0-7e0f-4ff9-bcee-b02b1a0e0be0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:20.075442 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e8c3a0-7e0f-4ff9-bcee-b02b1a0e0be0 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:20.101024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 901/1828] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:20.102313 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8c3a0-7e0f-4ff9-bcee-b02b1a0e0be0 service cinder] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:20.128642 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8c3a0-7e0f-4ff9-bcee-b02b1a0e0be0 service cinder] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:20.131717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e0fee573-dc77-40e2-a184-5f0df28a652c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '204f8f384bc54e699f26d5f9ba6bc590', 'request_id': 'req-e0fee573-dc77-40e2-a184-5f0df28a652c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:20.148588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e0fee573-dc77-40e2-a184-5f0df28a652c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '204f8f384bc54e699f26d5f9ba6bc590', 'request_id': 'req-e0fee573-dc77-40e2-a184-5f0df28a652c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:20.175962 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e0fee573-dc77-40e2-a184-5f0df28a652c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '204f8f384bc54e699f26d5f9ba6bc590', 'request_id': 'req-e0fee573-dc77-40e2-a184-5f0df28a652c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:20.195341 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e0fee573-dc77-40e2-a184-5f0df28a652c admin admin] The token cache is being invalidate because project 204f8f384bc54e699f26d5f9ba6bc590 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:05:20.195725 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e0fee573-dc77-40e2-a184-5f0df28a652c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:20.198037 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e0fee573-dc77-40e2-a184-5f0df28a652c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:20.206312 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 928/1829] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:05:19 2022] DELETE /identity/v3/projects/204f8f384bc54e699f26d5f9ba6bc590 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:20.208725 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fee573-dc77-40e2-a184-5f0df28a652c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:20.213054 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e0fee573-dc77-40e2-a184-5f0df28a652c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:20.302563 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb9103-5481-4693-ba35-bb4ff1ba3d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:20.303502 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb9103-5481-4693-ba35-bb4ff1ba3d65 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:20.303812 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb9103-5481-4693-ba35-bb4ff1ba3d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:20.304159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb9103-5481-4693-ba35-bb4ff1ba3d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:20.305268 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb9103-5481-4693-ba35-bb4ff1ba3d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:20.307494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb9103-5481-4693-ba35-bb4ff1ba3d65 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:20.343215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 902/1830] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:20.344195 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb9103-5481-4693-ba35-bb4ff1ba3d65 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:20.346103 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-7ceb9103-5481-4693-ba35-bb4ff1ba3d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:20.370009 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fee573-dc77-40e2-a184-5f0df28a652c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:20.378414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096c68f-be3a-4458-85b3-d6671872d52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:20.380319 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096c68f-be3a-4458-85b3-d6671872d52a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:20.380944 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096c68f-be3a-4458-85b3-d6671872d52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:20.381518 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096c68f-be3a-4458-85b3-d6671872d52a admin admin] PATH_INFO: `/v3/users/bd92aab18b924c738566dcd9dfe08c24` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:20.384332 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096c68f-be3a-4458-85b3-d6671872d52a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd92aab18b924c738566dcd9dfe08c24)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:20.388752 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096c68f-be3a-4458-85b3-d6671872d52a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:20.397307 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:20.434131 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:20.485805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb9103-5481-4693-ba35-bb4ff1ba3d65 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:20.498071 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3758fd-96a7-46a6-b0d0-48a2a87ed21c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:20.499146 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3758fd-96a7-46a6-b0d0-48a2a87ed21c service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:20.499562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3758fd-96a7-46a6-b0d0-48a2a87ed21c service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:20.499981 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4096c68f-be3a-4458-85b3-d6671872d52a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd92aab18b924c738566dcd9dfe08c24'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:20.500717 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3758fd-96a7-46a6-b0d0-48a2a87ed21c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:20.503203 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-2c3758fd-96a7-46a6-b0d0-48a2a87ed21c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:20.521389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4096c68f-be3a-4458-85b3-d6671872d52a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd92aab18b924c738566dcd9dfe08c24'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:20.553676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4096c68f-be3a-4458-85b3-d6671872d52a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd92aab18b924c738566dcd9dfe08c24'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:20.554717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4096c68f-be3a-4458-85b3-d6671872d52a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd92aab18b924c738566dcd9dfe08c24'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:20.555469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4096c68f-be3a-4458-85b3-d6671872d52a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd92aab18b924c738566dcd9dfe08c24'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:20.573094 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 929/1831] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:05:20 2022] DELETE /identity/v3/users/bd92aab18b924c738566dcd9dfe08c24 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:20.577140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096c68f-be3a-4458-85b3-d6671872d52a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:20.580085 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4096c68f-be3a-4458-85b3-d6671872d52a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:20.649301 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3758fd-96a7-46a6-b0d0-48a2a87ed21c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:20.650933 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3758fd-96a7-46a6-b0d0-48a2a87ed21c service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:20.673271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 903/1832] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:20.687617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096c68f-be3a-4458-85b3-d6671872d52a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:20.695013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fd33e-610f-4556-8fdf-cb862f2ee87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:20.696160 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fd33e-610f-4556-8fdf-cb862f2ee87e admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:20.696453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fd33e-610f-4556-8fdf-cb862f2ee87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:20.696731 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fd33e-610f-4556-8fdf-cb862f2ee87e admin admin] PATH_INFO: `/v3/projects/26130a4c94624fa1af487cee289237a8/users/a139ea014fc344e2a1f2ecfb6aa6fa51/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:20.736137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2fd33e-610f-4556-8fdf-cb862f2ee87e admin admin] RBAC: Authorizing `identity:create_grant(project_id=26130a4c94624fa1af487cee289237a8, user_id=a139ea014fc344e2a1f2ecfb6aa6fa51, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:20.742604 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2fd33e-610f-4556-8fdf-cb862f2ee87e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:20.772321 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 930/1833] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:20 2022] PUT /identity/v3/projects/26130a4c94624fa1af487cee289237a8/users/a139ea014fc344e2a1f2ecfb6aa6fa51/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:20.776413 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3758fd-96a7-46a6-b0d0-48a2a87ed21c service cinder] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:20.777910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-2c3758fd-96a7-46a6-b0d0-48a2a87ed21c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:20.809112 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3758fd-96a7-46a6-b0d0-48a2a87ed21c service cinder] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:20.946837 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8417b9c-7f56-4e3f-a8c0-3a9af61f13af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:20.947764 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8417b9c-7f56-4e3f-a8c0-3a9af61f13af admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:20.948299 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8417b9c-7f56-4e3f-a8c0-3a9af61f13af admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:20.948670 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8417b9c-7f56-4e3f-a8c0-3a9af61f13af admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:20.949801 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8417b9c-7f56-4e3f-a8c0-3a9af61f13af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:20.951724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8417b9c-7f56-4e3f-a8c0-3a9af61f13af admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:20.989284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 904/1834] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:20.993405 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fd33e-610f-4556-8fdf-cb862f2ee87e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:21.012106 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fd33e-610f-4556-8fdf-cb862f2ee87e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:21.026160 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2295fb-f1de-451a-a6df-481a56318146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:21.027237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2295fb-f1de-451a-a6df-481a56318146 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:21.027522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2295fb-f1de-451a-a6df-481a56318146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:21.028961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2295fb-f1de-451a-a6df-481a56318146 admin admin] PATH_INFO: `/v3/projects/26130a4c94624fa1af487cee289237a8/users/a139ea014fc344e2a1f2ecfb6aa6fa51/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:21.050704 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2295fb-f1de-451a-a6df-481a56318146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26130a4c94624fa1af487cee289237a8, user_id=a139ea014fc344e2a1f2ecfb6aa6fa51, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:21.053483 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2295fb-f1de-451a-a6df-481a56318146 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:21.081258 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 931/1835] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:20 2022] PUT /identity/v3/projects/26130a4c94624fa1af487cee289237a8/users/a139ea014fc344e2a1f2ecfb6aa6fa51/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:21.091898 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a7630-e9b5-4132-b053-4280eb3eb48f None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:21.092288 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a7630-e9b5-4132-b053-4280eb3eb48f None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:21.092522 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a7630-e9b5-4132-b053-4280eb3eb48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:21.133115 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2295fb-f1de-451a-a6df-481a56318146 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:21.137042 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-de2295fb-f1de-451a-a6df-481a56318146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:21.168681 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-630a7630-e9b5-4132-b053-4280eb3eb48f None None] MFA Rules not processed for user `a139ea014fc344e2a1f2ecfb6aa6fa51`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:21.169940 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-630a7630-e9b5-4132-b053-4280eb3eb48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:21.272560 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:21.323068 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2295fb-f1de-451a-a6df-481a56318146 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:21.336974 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d2e24-fe80-4fcd-a8bd-5c5daf037e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:21.338171 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d2e24-fe80-4fcd-a8bd-5c5daf037e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:21.342053 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d2e24-fe80-4fcd-a8bd-5c5daf037e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:21.342597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d2e24-fe80-4fcd-a8bd-5c5daf037e68 admin admin] PATH_INFO: `/v3/users/650b4f20681444c5b85ba22c9424b05b` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:21.343691 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 905/1836] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:21 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:21.356914 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d2e24-fe80-4fcd-a8bd-5c5daf037e68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=650b4f20681444c5b85ba22c9424b05b)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:21.363731 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a7630-e9b5-4132-b053-4280eb3eb48f None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:21.364117 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d2e24-fe80-4fcd-a8bd-5c5daf037e68 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:21.366048 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-630a7630-e9b5-4132-b053-4280eb3eb48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:21.373393 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:21.492622 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-244d2e24-fe80-4fcd-a8bd-5c5daf037e68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '650b4f20681444c5b85ba22c9424b05b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:21.504129 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-244d2e24-fe80-4fcd-a8bd-5c5daf037e68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '650b4f20681444c5b85ba22c9424b05b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:21.508135 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a7630-e9b5-4132-b053-4280eb3eb48f None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:21.516638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e379b-b2a9-4879-a0a4-1e39a32e9c75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:21.517337 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e379b-b2a9-4879-a0a4-1e39a32e9c75 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:21.517626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e379b-b2a9-4879-a0a4-1e39a32e9c75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:21.517921 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e379b-b2a9-4879-a0a4-1e39a32e9c75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:21.529757 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e379b-b2a9-4879-a0a4-1e39a32e9c75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:21.532190 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e379b-b2a9-4879-a0a4-1e39a32e9c75 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:21.535130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-244d2e24-fe80-4fcd-a8bd-5c5daf037e68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '650b4f20681444c5b85ba22c9424b05b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:21.536772 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-244d2e24-fe80-4fcd-a8bd-5c5daf037e68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '650b4f20681444c5b85ba22c9424b05b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:21.537977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-244d2e24-fe80-4fcd-a8bd-5c5daf037e68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '650b4f20681444c5b85ba22c9424b05b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:21.545299 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 906/1837] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:21 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:05:21.563823 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 932/1838] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:05:21 2022] DELETE /identity/v3/users/650b4f20681444c5b85ba22c9424b05b => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:21.569219 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5acd3f-002d-4c02-b569-d58e3a0d7d2a None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:21.569601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5acd3f-002d-4c02-b569-d58e3a0d7d2a None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:21.569871 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5acd3f-002d-4c02-b569-d58e3a0d7d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:21.595214 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-6e5acd3f-002d-4c02-b569-d58e3a0d7d2a None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:21.596573 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-6e5acd3f-002d-4c02-b569-d58e3a0d7d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:21.662975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d2e24-fe80-4fcd-a8bd-5c5daf037e68 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:21.664776 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-244d2e24-fe80-4fcd-a8bd-5c5daf037e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:21.715286 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 907/1839] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:21.772870 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d2e24-fe80-4fcd-a8bd-5c5daf037e68 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:21.777797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a3779-2272-4931-8127-22debdd12d53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:21.778693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a3779-2272-4931-8127-22debdd12d53 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:21.779073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a3779-2272-4931-8127-22debdd12d53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:21.779501 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a3779-2272-4931-8127-22debdd12d53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:21.783310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5acd3f-002d-4c02-b569-d58e3a0d7d2a None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:21.787494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-6e5acd3f-002d-4c02-b569-d58e3a0d7d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:21.788622 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-015a3779-2272-4931-8127-22debdd12d53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:21.880363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5acd3f-002d-4c02-b569-d58e3a0d7d2a None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:21.887528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf80a1-77af-4407-936b-5266791d4304 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:21.888344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf80a1-77af-4407-936b-5266791d4304 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:21.888767 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf80a1-77af-4407-936b-5266791d4304 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:21.889331 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf80a1-77af-4407-936b-5266791d4304 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:21.907294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf80a1-77af-4407-936b-5266791d4304 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:21.908933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a3779-2272-4931-8127-22debdd12d53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:21.909865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf80a1-77af-4407-936b-5266791d4304 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:21.910442 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a3779-2272-4931-8127-22debdd12d53 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:21.926812 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 908/1840] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:21.936251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 933/1841] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:21 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:05:22.137333 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf95039-3ef2-4ff2-b6b7-4a4fe3ce0fc5 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:22.154181 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf95039-3ef2-4ff2-b6b7-4a4fe3ce0fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:22.154589 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf95039-3ef2-4ff2-b6b7-4a4fe3ce0fc5 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:22.162408 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 909/1842] 10.222.0.76 () {54 vars in 921 bytes} [Thu May 26 18:05:22 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:05:22.176775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f6fdc-7f97-46bc-9809-eb1f9410d230 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:22.176775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f6fdc-7f97-46bc-9809-eb1f9410d230 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:22.177462 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f6fdc-7f97-46bc-9809-eb1f9410d230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:22.209367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0b0f6fdc-7f97-46bc-9809-eb1f9410d230 None None] MFA Rules not processed for user `1899c317083d40f5b490cd9eb63fe521`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:22.213095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0b0f6fdc-7f97-46bc-9809-eb1f9410d230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:22.243723 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:22.319000 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 934/1843] 10.222.0.76 () {58 vars in 1018 bytes} [Thu May 26 18:05:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:22.377139 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf95039-3ef2-4ff2-b6b7-4a4fe3ce0fc5 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:22.378814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-edf95039-3ef2-4ff2-b6b7-4a4fe3ce0fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:22.399696 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf95039-3ef2-4ff2-b6b7-4a4fe3ce0fc5 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:22.405809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71951b-2b9a-4c3b-8a22-511d45f6cc4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:22.406561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71951b-2b9a-4c3b-8a22-511d45f6cc4f service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:22.407005 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71951b-2b9a-4c3b-8a22-511d45f6cc4f service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:22.407288 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71951b-2b9a-4c3b-8a22-511d45f6cc4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:22.416777 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af71951b-2b9a-4c3b-8a22-511d45f6cc4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:22.418409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af71951b-2b9a-4c3b-8a22-511d45f6cc4f service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:22.429655 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 910/1844] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:22.785505 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f6fdc-7f97-46bc-9809-eb1f9410d230 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:22.786981 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0b0f6fdc-7f97-46bc-9809-eb1f9410d230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:22.800763 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f6fdc-7f97-46bc-9809-eb1f9410d230 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:22.805671 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90238b5f-97ec-430b-b3d1-34b237173f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:22.806597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90238b5f-97ec-430b-b3d1-34b237173f18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:22.806879 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90238b5f-97ec-430b-b3d1-34b237173f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:22.807152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90238b5f-97ec-430b-b3d1-34b237173f18 admin admin] PATH_INFO: `/v3/projects/0ed06a764de944f09e6656b70280f04d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:22.808409 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90238b5f-97ec-430b-b3d1-34b237173f18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ed06a764de944f09e6656b70280f04d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:22.810419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90238b5f-97ec-430b-b3d1-34b237173f18 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:22.862151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-90238b5f-97ec-430b-b3d1-34b237173f18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ed06a764de944f09e6656b70280f04d', 'request_id': 'req-90238b5f-97ec-430b-b3d1-34b237173f18'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:22.868179 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-90238b5f-97ec-430b-b3d1-34b237173f18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ed06a764de944f09e6656b70280f04d', 'request_id': 'req-90238b5f-97ec-430b-b3d1-34b237173f18'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:22.874687 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-90238b5f-97ec-430b-b3d1-34b237173f18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ed06a764de944f09e6656b70280f04d', 'request_id': 'req-90238b5f-97ec-430b-b3d1-34b237173f18'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:22.890930 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-90238b5f-97ec-430b-b3d1-34b237173f18 admin admin] The token cache is being invalidate because project 0ed06a764de944f09e6656b70280f04d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:05:22.891197 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-90238b5f-97ec-430b-b3d1-34b237173f18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:22.892011 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-90238b5f-97ec-430b-b3d1-34b237173f18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:22.893815 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 935/1845] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:05:22 2022] DELETE /identity/v3/projects/0ed06a764de944f09e6656b70280f04d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:22.900084 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2b9fb-3970-480d-ad6d-0f2560835e71 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:22.900827 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2b9fb-3970-480d-ad6d-0f2560835e71 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:22.900827 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2b9fb-3970-480d-ad6d-0f2560835e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:22.925618 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-63d2b9fb-3970-480d-ad6d-0f2560835e71 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:22.927141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-63d2b9fb-3970-480d-ad6d-0f2560835e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:22.963844 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:22.999126 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 911/1846] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:23.009822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2936b565-8cc8-4ee8-b322-c8f45cc44714 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:23.010139 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2936b565-8cc8-4ee8-b322-c8f45cc44714 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:23.010525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2936b565-8cc8-4ee8-b322-c8f45cc44714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:23.036008 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2936b565-8cc8-4ee8-b322-c8f45cc44714 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:23.037556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2936b565-8cc8-4ee8-b322-c8f45cc44714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:23.046785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 936/1847] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:23.051524 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2b9fb-3970-480d-ad6d-0f2560835e71 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:23.061948 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2b9fb-3970-480d-ad6d-0f2560835e71 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:23.068459 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b68bba-6aa9-4fe3-966f-584d59c4c343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:23.069012 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b68bba-6aa9-4fe3-966f-584d59c4c343 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:23.069297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b68bba-6aa9-4fe3-966f-584d59c4c343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:23.069574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b68bba-6aa9-4fe3-966f-584d59c4c343 admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:23.070354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b68bba-6aa9-4fe3-966f-584d59c4c343 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:23.070659 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b68bba-6aa9-4fe3-966f-584d59c4c343 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:23.072270 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b68bba-6aa9-4fe3-966f-584d59c4c343 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:23.085678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 912/1848] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:05:23.090597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2936b565-8cc8-4ee8-b322-c8f45cc44714 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:23.100779 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2936b565-8cc8-4ee8-b322-c8f45cc44714 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:23.105593 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68cacd-0f99-4ebf-b943-081e08308401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:23.106292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68cacd-0f99-4ebf-b943-081e08308401 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:23.106582 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68cacd-0f99-4ebf-b943-081e08308401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:23.106936 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68cacd-0f99-4ebf-b943-081e08308401 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:23.107608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea68cacd-0f99-4ebf-b943-081e08308401 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:23.110070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea68cacd-0f99-4ebf-b943-081e08308401 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:23.140142 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 937/1849] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:05:23 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:23.145130 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b68bba-6aa9-4fe3-966f-584d59c4c343 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:23.171014 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b68bba-6aa9-4fe3-966f-584d59c4c343 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:23.236655 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fa18c-7dea-4e36-8cb3-b26d5dfe75f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:23.237939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fa18c-7dea-4e36-8cb3-b26d5dfe75f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:23.238056 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fa18c-7dea-4e36-8cb3-b26d5dfe75f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:23.238380 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fa18c-7dea-4e36-8cb3-b26d5dfe75f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:23.239086 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081fa18c-7dea-4e36-8cb3-b26d5dfe75f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:23.241504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081fa18c-7dea-4e36-8cb3-b26d5dfe75f0 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:23.265085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 913/1850] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:05:23 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:23.266705 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68cacd-0f99-4ebf-b943-081e08308401 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:23.276857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68cacd-0f99-4ebf-b943-081e08308401 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:23.282583 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5062d9-c7d3-4dd2-bc86-cb5367f9c9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:23.283265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5062d9-c7d3-4dd2-bc86-cb5367f9c9df admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:23.283553 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5062d9-c7d3-4dd2-bc86-cb5367f9c9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:23.283835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5062d9-c7d3-4dd2-bc86-cb5367f9c9df admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:23.285136 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5062d9-c7d3-4dd2-bc86-cb5367f9c9df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:23.286428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5062d9-c7d3-4dd2-bc86-cb5367f9c9df admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:23.317445 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 938/1851] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:23.320729 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fa18c-7dea-4e36-8cb3-b26d5dfe75f0 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:23.322303 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5062d9-c7d3-4dd2-bc86-cb5367f9c9df admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:23.325169 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8b5062d9-c7d3-4dd2-bc86-cb5367f9c9df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:23.341155 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fa18c-7dea-4e36-8cb3-b26d5dfe75f0 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:23.347862 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5062d9-c7d3-4dd2-bc86-cb5367f9c9df admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:23.352970 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75edb3-792d-4228-8b0b-96803877473e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:23.353851 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75edb3-792d-4228-8b0b-96803877473e admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:23.354177 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0caaae-ce7e-4734-8ee4-89919ce2b6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:23.354503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75edb3-792d-4228-8b0b-96803877473e admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:23.354934 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75edb3-792d-4228-8b0b-96803877473e admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:23.355383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0caaae-ce7e-4734-8ee4-89919ce2b6bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:23.355764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0caaae-ce7e-4734-8ee4-89919ce2b6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:23.356137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0caaae-ce7e-4734-8ee4-89919ce2b6bd admin admin] PATH_INFO: `/v3/projects/eae794b2af6b47cc8b3efd9142467f2f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:23.356444 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75edb3-792d-4228-8b0b-96803877473e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:23.357580 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0caaae-ce7e-4734-8ee4-89919ce2b6bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=eae794b2af6b47cc8b3efd9142467f2f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:23.358481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75edb3-792d-4228-8b0b-96803877473e admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:23.361002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0caaae-ce7e-4734-8ee4-89919ce2b6bd admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:23.373155 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 914/1852] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:23.378846 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75edb3-792d-4228-8b0b-96803877473e admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:23.396490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75edb3-792d-4228-8b0b-96803877473e admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:23.410379 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182a4b6-4842-47b9-a057-edd201edf411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:23.411407 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182a4b6-4842-47b9-a057-edd201edf411 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:23.411697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182a4b6-4842-47b9-a057-edd201edf411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:23.411983 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182a4b6-4842-47b9-a057-edd201edf411 admin admin] PATH_INFO: `/v3/projects/6d55e5af817446ff85e70f08c4bf3e89/users/fe7f81548cb34a509d2eca66fe943861/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:23.444383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182a4b6-4842-47b9-a057-edd201edf411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d55e5af817446ff85e70f08c4bf3e89, user_id=fe7f81548cb34a509d2eca66fe943861, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:23.458041 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182a4b6-4842-47b9-a057-edd201edf411 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:23.466383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cb0caaae-ce7e-4734-8ee4-89919ce2b6bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eae794b2af6b47cc8b3efd9142467f2f', 'request_id': 'req-cb0caaae-ce7e-4734-8ee4-89919ce2b6bd'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:23.473835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:23.490169 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cb0caaae-ce7e-4734-8ee4-89919ce2b6bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eae794b2af6b47cc8b3efd9142467f2f', 'request_id': 'req-cb0caaae-ce7e-4734-8ee4-89919ce2b6bd'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:23.503278 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 915/1853] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:23 2022] PUT /identity/v3/projects/6d55e5af817446ff85e70f08c4bf3e89/users/fe7f81548cb34a509d2eca66fe943861/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:05:23.505700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea8195-5bb4-417a-9697-6b1baa8e9616 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:23.506084 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea8195-5bb4-417a-9697-6b1baa8e9616 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:23.506398 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea8195-5bb4-417a-9697-6b1baa8e9616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:23.510077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cb0caaae-ce7e-4734-8ee4-89919ce2b6bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eae794b2af6b47cc8b3efd9142467f2f', 'request_id': 'req-cb0caaae-ce7e-4734-8ee4-89919ce2b6bd'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:23.533029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cb0caaae-ce7e-4734-8ee4-89919ce2b6bd admin admin] The token cache is being invalidate because project eae794b2af6b47cc8b3efd9142467f2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:05:23.533333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cb0caaae-ce7e-4734-8ee4-89919ce2b6bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:23.534120 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cb0caaae-ce7e-4734-8ee4-89919ce2b6bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:23.536882 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 939/1854] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:05:23 2022] DELETE /identity/v3/projects/eae794b2af6b47cc8b3efd9142467f2f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:23.546248 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0caaae-ce7e-4734-8ee4-89919ce2b6bd admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:23.548422 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cb0caaae-ce7e-4734-8ee4-89919ce2b6bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:23.553138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-54ea8195-5bb4-417a-9697-6b1baa8e9616 None None] MFA Rules not processed for user `fe7f81548cb34a509d2eca66fe943861`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:23.555197 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-54ea8195-5bb4-417a-9697-6b1baa8e9616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:23.689691 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 916/1855] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:23.695585 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea8195-5bb4-417a-9697-6b1baa8e9616 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:23.698723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-54ea8195-5bb4-417a-9697-6b1baa8e9616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:23.716098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:23.739380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0caaae-ce7e-4734-8ee4-89919ce2b6bd admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:23.813487 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea8195-5bb4-417a-9697-6b1baa8e9616 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:23.821470 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ce09c-cb67-4e42-835e-9d44c9b90e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:23.822135 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ce09c-cb67-4e42-835e-9d44c9b90e01 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:23.822576 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ce09c-cb67-4e42-835e-9d44c9b90e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:23.822863 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ce09c-cb67-4e42-835e-9d44c9b90e01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:23.823590 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5ce09c-cb67-4e42-835e-9d44c9b90e01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:23.826396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5ce09c-cb67-4e42-835e-9d44c9b90e01 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:23.844914 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a251ea4a-cf62-4c36-aeef-d4a248759a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:23.846347 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251ea4a-cf62-4c36-aeef-d4a248759a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:23.846742 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251ea4a-cf62-4c36-aeef-d4a248759a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:23.847125 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251ea4a-cf62-4c36-aeef-d4a248759a84 admin admin] PATH_INFO: `/v3/users/306e8ec162d04074906fd9ec1b9e936e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:23.853822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a251ea4a-cf62-4c36-aeef-d4a248759a84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=306e8ec162d04074906fd9ec1b9e936e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:23.855575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a251ea4a-cf62-4c36-aeef-d4a248759a84 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:23.897980 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 917/1856] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:05:23 2022] POST /identity/v3/projects => generated 393 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:23.899577 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ce09c-cb67-4e42-835e-9d44c9b90e01 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:23.957777 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a251ea4a-cf62-4c36-aeef-d4a248759a84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '306e8ec162d04074906fd9ec1b9e936e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:23.989112 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a251ea4a-cf62-4c36-aeef-d4a248759a84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '306e8ec162d04074906fd9ec1b9e936e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:24.006103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a251ea4a-cf62-4c36-aeef-d4a248759a84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '306e8ec162d04074906fd9ec1b9e936e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:24.006934 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a251ea4a-cf62-4c36-aeef-d4a248759a84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '306e8ec162d04074906fd9ec1b9e936e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:24.007666 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a251ea4a-cf62-4c36-aeef-d4a248759a84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '306e8ec162d04074906fd9ec1b9e936e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:24.046055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ce09c-cb67-4e42-835e-9d44c9b90e01 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:24.054505 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 940/1857] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:05:23 2022] DELETE /identity/v3/users/306e8ec162d04074906fd9ec1b9e936e => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:24.082254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d62ac-a4dc-4fe5-bdea-7b55c1d91541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:24.082967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d62ac-a4dc-4fe5-bdea-7b55c1d91541 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:24.083410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d62ac-a4dc-4fe5-bdea-7b55c1d91541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:24.083608 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d62ac-a4dc-4fe5-bdea-7b55c1d91541 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:24.084333 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d62ac-a4dc-4fe5-bdea-7b55c1d91541 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:24.086340 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d62ac-a4dc-4fe5-bdea-7b55c1d91541 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:24.117021 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 918/1858] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:05:23 2022] POST /identity/v3/users => generated 303 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:24.121777 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a251ea4a-cf62-4c36-aeef-d4a248759a84 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:24.123551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a251ea4a-cf62-4c36-aeef-d4a248759a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:24.159085 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:24.292115 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a251ea4a-cf62-4c36-aeef-d4a248759a84 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:24.322225 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6e25c-2174-4c37-98e1-ae8be0911f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:24.322982 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6e25c-2174-4c37-98e1-ae8be0911f8e admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:24.323300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6e25c-2174-4c37-98e1-ae8be0911f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:24.323583 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6e25c-2174-4c37-98e1-ae8be0911f8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:24.325094 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd6e25c-2174-4c37-98e1-ae8be0911f8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:24.327676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd6e25c-2174-4c37-98e1-ae8be0911f8e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:24.346248 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 941/1859] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:24.348253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d62ac-a4dc-4fe5-bdea-7b55c1d91541 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:24.360441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d62ac-a4dc-4fe5-bdea-7b55c1d91541 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:24.365733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd990e87-7ccb-48cc-a0cf-2c06713db597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:24.366669 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd990e87-7ccb-48cc-a0cf-2c06713db597 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:24.366957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd990e87-7ccb-48cc-a0cf-2c06713db597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:24.367220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd990e87-7ccb-48cc-a0cf-2c06713db597 admin admin] PATH_INFO: `/v3/projects/cb33d2a9c51b40818ba8223009114518/users/4d46ddd3669941ab967a38a15a6eb8a9/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:24.400781 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd990e87-7ccb-48cc-a0cf-2c06713db597 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb33d2a9c51b40818ba8223009114518, user_id=4d46ddd3669941ab967a38a15a6eb8a9, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:24.403562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd990e87-7ccb-48cc-a0cf-2c06713db597 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:24.425143 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 919/1860] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:24 2022] PUT /identity/v3/projects/cb33d2a9c51b40818ba8223009114518/users/4d46ddd3669941ab967a38a15a6eb8a9/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:24.429376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6e25c-2174-4c37-98e1-ae8be0911f8e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:24.447251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6e25c-2174-4c37-98e1-ae8be0911f8e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:24.462766 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd990e87-7ccb-48cc-a0cf-2c06713db597 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:24.464806 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-dd990e87-7ccb-48cc-a0cf-2c06713db597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:24.518643 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2a3aa-8062-4101-af68-1da08c1b3c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:24.519331 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2a3aa-8062-4101-af68-1da08c1b3c7c admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:24.519620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2a3aa-8062-4101-af68-1da08c1b3c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:24.519900 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2a3aa-8062-4101-af68-1da08c1b3c7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:24.520627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2a3aa-8062-4101-af68-1da08c1b3c7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:24.522294 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2a3aa-8062-4101-af68-1da08c1b3c7c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:24.534532 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 942/1861] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:24.538964 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2a3aa-8062-4101-af68-1da08c1b3c7c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:24.545913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:24.551725 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd990e87-7ccb-48cc-a0cf-2c06713db597 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:24.553063 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2a3aa-8062-4101-af68-1da08c1b3c7c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:24.560278 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6a892-db82-42cc-9f79-833f863fe7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:24.561168 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c08865-035e-4321-b280-cdfeb0948b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:24.561374 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6a892-db82-42cc-9f79-833f863fe7a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:24.561551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6a892-db82-42cc-9f79-833f863fe7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:24.561792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6a892-db82-42cc-9f79-833f863fe7a7 admin admin] PATH_INFO: `/v3/projects/cb33d2a9c51b40818ba8223009114518/users/4d46ddd3669941ab967a38a15a6eb8a9/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:24.562450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c08865-035e-4321-b280-cdfeb0948b76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:24.563506 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c08865-035e-4321-b280-cdfeb0948b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:24.563910 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c08865-035e-4321-b280-cdfeb0948b76 admin admin] PATH_INFO: `/v3/projects/50ed3c4298544707945fda2d53e8112a` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:24.564500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6a892-db82-42cc-9f79-833f863fe7a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb33d2a9c51b40818ba8223009114518, user_id=4d46ddd3669941ab967a38a15a6eb8a9, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:24.565856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c08865-035e-4321-b280-cdfeb0948b76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50ed3c4298544707945fda2d53e8112a)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:24.567191 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6a892-db82-42cc-9f79-833f863fe7a7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:24.567878 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c08865-035e-4321-b280-cdfeb0948b76 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:24.580785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 943/1862] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:24 2022] PUT /identity/v3/projects/cb33d2a9c51b40818ba8223009114518/users/4d46ddd3669941ab967a38a15a6eb8a9/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:05:24.590599 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e4f6d-0afe-4b84-b950-53d80fe95669 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:24.591463 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e4f6d-0afe-4b84-b950-53d80fe95669 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:24.591463 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e4f6d-0afe-4b84-b950-53d80fe95669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:24.631390 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2a8e4f6d-0afe-4b84-b950-53d80fe95669 None None] MFA Rules not processed for user `4d46ddd3669941ab967a38a15a6eb8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:24.632901 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2a8e4f6d-0afe-4b84-b950-53d80fe95669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:24.644030 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-e5c08865-035e-4321-b280-cdfeb0948b76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50ed3c4298544707945fda2d53e8112a', 'request_id': 'req-e5c08865-035e-4321-b280-cdfeb0948b76'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:24.653326 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-e5c08865-035e-4321-b280-cdfeb0948b76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50ed3c4298544707945fda2d53e8112a', 'request_id': 'req-e5c08865-035e-4321-b280-cdfeb0948b76'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:24.663506 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-e5c08865-035e-4321-b280-cdfeb0948b76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50ed3c4298544707945fda2d53e8112a', 'request_id': 'req-e5c08865-035e-4321-b280-cdfeb0948b76'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:24.677542 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-e5c08865-035e-4321-b280-cdfeb0948b76 admin admin] The token cache is being invalidate because project 50ed3c4298544707945fda2d53e8112a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:05:24.678042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-e5c08865-035e-4321-b280-cdfeb0948b76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:24.678853 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-e5c08865-035e-4321-b280-cdfeb0948b76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:24.684171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 920/1863] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:05:24 2022] DELETE /identity/v3/projects/50ed3c4298544707945fda2d53e8112a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:24.685697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fdfe7-54a5-48c5-8f89-598f333b15e9 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:24.685980 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fdfe7-54a5-48c5-8f89-598f333b15e9 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:24.686483 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fdfe7-54a5-48c5-8f89-598f333b15e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:24.708748 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-ea8fdfe7-54a5-48c5-8f89-598f333b15e9 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:24.710480 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-ea8fdfe7-54a5-48c5-8f89-598f333b15e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:24.716561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 944/1864] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:24 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:05:24.729458 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e4f6d-0afe-4b84-b950-53d80fe95669 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:24.731169 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2a8e4f6d-0afe-4b84-b950-53d80fe95669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:24.752281 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:24.788418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 921/1865] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:24.798576 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f34f2e-bf90-4d25-97d9-dae7127ccb18 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:24.799163 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f34f2e-bf90-4d25-97d9-dae7127ccb18 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:24.799586 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f34f2e-bf90-4d25-97d9-dae7127ccb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:24.804636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e4f6d-0afe-4b84-b950-53d80fe95669 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:24.808993 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976249e-aed4-4c6b-ac15-1dd95223085d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:24.809667 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976249e-aed4-4c6b-ac15-1dd95223085d service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:24.809955 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976249e-aed4-4c6b-ac15-1dd95223085d service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:24.810266 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976249e-aed4-4c6b-ac15-1dd95223085d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:24.825927 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-29f34f2e-bf90-4d25-97d9-dae7127ccb18 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:24.827211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-29f34f2e-bf90-4d25-97d9-dae7127ccb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:24.836922 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 922/1866] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:24.841029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f34f2e-bf90-4d25-97d9-dae7127ccb18 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:24.841565 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976249e-aed4-4c6b-ac15-1dd95223085d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:24.845215 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976249e-aed4-4c6b-ac15-1dd95223085d service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:24.852513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f34f2e-bf90-4d25-97d9-dae7127ccb18 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:24.857272 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 945/1867] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:24 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:05:24.858401 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62f153-7ac7-4fed-951c-c135ab15e9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:24.859058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62f153-7ac7-4fed-951c-c135ab15e9be admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:24.859486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62f153-7ac7-4fed-951c-c135ab15e9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:24.860320 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62f153-7ac7-4fed-951c-c135ab15e9be admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:24.861071 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62f153-7ac7-4fed-951c-c135ab15e9be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:24.861384 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62f153-7ac7-4fed-951c-c135ab15e9be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:24.863103 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62f153-7ac7-4fed-951c-c135ab15e9be admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:24.878369 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 923/1868] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:24.882565 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976249e-aed4-4c6b-ac15-1dd95223085d service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:24.896945 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976249e-aed4-4c6b-ac15-1dd95223085d service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:24.913744 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b261530-f54d-4800-84e8-878d7a313ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:24.915694 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b261530-f54d-4800-84e8-878d7a313ff7 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:24.915997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b261530-f54d-4800-84e8-878d7a313ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:24.917727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b261530-f54d-4800-84e8-878d7a313ff7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:24.918224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b261530-f54d-4800-84e8-878d7a313ff7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:24.920889 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b261530-f54d-4800-84e8-878d7a313ff7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:24.952724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62f153-7ac7-4fed-951c-c135ab15e9be admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:24.965048 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62f153-7ac7-4fed-951c-c135ab15e9be admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:24.970708 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 946/1869] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:05:24 2022] POST /identity/v3/projects => generated 375 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:24.972400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa7465-7b95-4558-93ab-7a24393a152f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:24.973537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa7465-7b95-4558-93ab-7a24393a152f service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:24.974085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa7465-7b95-4558-93ab-7a24393a152f service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:24.974643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa7465-7b95-4558-93ab-7a24393a152f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:24.975874 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b261530-f54d-4800-84e8-878d7a313ff7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:24.977190 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-6dfa7465-7b95-4558-93ab-7a24393a152f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:25.061889 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b261530-f54d-4800-84e8-878d7a313ff7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:25.071605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebd2c8-2c1a-4733-8155-b33dad120532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:25.073154 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebd2c8-2c1a-4733-8155-b33dad120532 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:25.073933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebd2c8-2c1a-4733-8155-b33dad120532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:25.074573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebd2c8-2c1a-4733-8155-b33dad120532 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:25.076343 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebd2c8-2c1a-4733-8155-b33dad120532 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:25.082841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebd2c8-2c1a-4733-8155-b33dad120532 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:25.090405 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfa7465-7b95-4558-93ab-7a24393a152f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:25.094804 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfa7465-7b95-4558-93ab-7a24393a152f service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:25.111513 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 947/1870] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:05:24 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:25.112056 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 924/1871] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:05:25.112875 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebd2c8-2c1a-4733-8155-b33dad120532 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:25.137514 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebd2c8-2c1a-4733-8155-b33dad120532 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:25.153632 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc174d1-2dc2-4fea-ae95-da22466ec2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:25.154546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc174d1-2dc2-4fea-ae95-da22466ec2ef admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:25.155037 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc174d1-2dc2-4fea-ae95-da22466ec2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:25.155689 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc174d1-2dc2-4fea-ae95-da22466ec2ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:25.157622 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc174d1-2dc2-4fea-ae95-da22466ec2ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:25.160107 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc174d1-2dc2-4fea-ae95-da22466ec2ef admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:25.186386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 948/1872] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:25.189270 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa7465-7b95-4558-93ab-7a24393a152f service cinder] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:25.209393 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa7465-7b95-4558-93ab-7a24393a152f service cinder] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:25.224072 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997423a8-b20c-4978-aac4-1f790aca721a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:25.225058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997423a8-b20c-4978-aac4-1f790aca721a admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:25.227758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997423a8-b20c-4978-aac4-1f790aca721a admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:25.228097 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997423a8-b20c-4978-aac4-1f790aca721a admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:25.228864 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997423a8-b20c-4978-aac4-1f790aca721a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:25.230525 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997423a8-b20c-4978-aac4-1f790aca721a admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:25.237504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:25.254738 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 925/1873] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:25.259344 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc174d1-2dc2-4fea-ae95-da22466ec2ef admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:25.278124 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc174d1-2dc2-4fea-ae95-da22466ec2ef admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:25.289205 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25724d0e-9261-42bd-8e44-3194614ff0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:25.290466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25724d0e-9261-42bd-8e44-3194614ff0af admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:25.290792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25724d0e-9261-42bd-8e44-3194614ff0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:25.291289 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25724d0e-9261-42bd-8e44-3194614ff0af admin admin] PATH_INFO: `/v3/projects/8cbe6d45be104dfaacc3afd241b940ca/users/4c6297456efe476e9a656e1a52f361d4/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:25.353982 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25724d0e-9261-42bd-8e44-3194614ff0af admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cbe6d45be104dfaacc3afd241b940ca, user_id=4c6297456efe476e9a656e1a52f361d4, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:25.356840 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25724d0e-9261-42bd-8e44-3194614ff0af admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:25.389922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 949/1874] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:25 2022] PUT /identity/v3/projects/8cbe6d45be104dfaacc3afd241b940ca/users/4c6297456efe476e9a656e1a52f361d4/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:25.408561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485611ee-3cd3-46a8-8d0c-75b2d26d16d9 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:25.408860 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485611ee-3cd3-46a8-8d0c-75b2d26d16d9 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:25.409171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485611ee-3cd3-46a8-8d0c-75b2d26d16d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:25.476690 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-485611ee-3cd3-46a8-8d0c-75b2d26d16d9 None None] MFA Rules not processed for user `4c6297456efe476e9a656e1a52f361d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:25.478053 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-485611ee-3cd3-46a8-8d0c-75b2d26d16d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:25.619486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 926/1875] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:25.633893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25724d0e-9261-42bd-8e44-3194614ff0af admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:25.639519 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-25724d0e-9261-42bd-8e44-3194614ff0af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:25.722625 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25724d0e-9261-42bd-8e44-3194614ff0af admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:25.729513 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7de9cc-1ac4-42a0-909f-09d8af3af598 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f4c9fc949c4a2c951b755234848cc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:25.730569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7de9cc-1ac4-42a0-909f-09d8af3af598 service glance] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:25.731123 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7de9cc-1ac4-42a0-909f-09d8af3af598 service glance] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:25.731611 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7de9cc-1ac4-42a0-909f-09d8af3af598 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:25.739887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:25.747935 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7de9cc-1ac4-42a0-909f-09d8af3af598 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:25.750813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7de9cc-1ac4-42a0-909f-09d8af3af598 service glance] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:25.781485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 950/1876] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:29.190231 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485611ee-3cd3-46a8-8d0c-75b2d26d16d9 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:29.191780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-485611ee-3cd3-46a8-8d0c-75b2d26d16d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:29.287153 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485611ee-3cd3-46a8-8d0c-75b2d26d16d9 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:29.294925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262157a1-75b8-48a9-ad00-db1bbef84c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:29.296134 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262157a1-75b8-48a9-ad00-db1bbef84c70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:29.296643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262157a1-75b8-48a9-ad00-db1bbef84c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:29.297691 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262157a1-75b8-48a9-ad00-db1bbef84c70 admin admin] PATH_INFO: `/v3/users/fe7f81548cb34a509d2eca66fe943861` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:29.301193 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262157a1-75b8-48a9-ad00-db1bbef84c70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe7f81548cb34a509d2eca66fe943861)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:29.304782 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262157a1-75b8-48a9-ad00-db1bbef84c70 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:29.375542 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-262157a1-75b8-48a9-ad00-db1bbef84c70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe7f81548cb34a509d2eca66fe943861'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:29.386989 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-262157a1-75b8-48a9-ad00-db1bbef84c70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe7f81548cb34a509d2eca66fe943861'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:29.388052 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-262157a1-75b8-48a9-ad00-db1bbef84c70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe7f81548cb34a509d2eca66fe943861'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:29.388759 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-262157a1-75b8-48a9-ad00-db1bbef84c70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe7f81548cb34a509d2eca66fe943861'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:29.401236 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-262157a1-75b8-48a9-ad00-db1bbef84c70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe7f81548cb34a509d2eca66fe943861'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:29.426385 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 927/1877] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:05:29 2022] DELETE /identity/v3/users/fe7f81548cb34a509d2eca66fe943861 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:29.431007 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242e8f7-b8be-4fb9-b0bd-e7ce793f38be None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:29.431995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242e8f7-b8be-4fb9-b0bd-e7ce793f38be None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:29.433218 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242e8f7-b8be-4fb9-b0bd-e7ce793f38be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:29.454234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-7242e8f7-b8be-4fb9-b0bd-e7ce793f38be None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:29.455703 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7242e8f7-b8be-4fb9-b0bd-e7ce793f38be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:29.543030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 951/1878] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:29.555467 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262157a1-75b8-48a9-ad00-db1bbef84c70 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:29.558215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-262157a1-75b8-48a9-ad00-db1bbef84c70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:29.571604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:29.643292 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262157a1-75b8-48a9-ad00-db1bbef84c70 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:29.650659 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0337f-5ae6-41ff-8eed-9c671094dad0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:29.651419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0337f-5ae6-41ff-8eed-9c671094dad0 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:29.651809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0337f-5ae6-41ff-8eed-9c671094dad0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:29.652558 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0337f-5ae6-41ff-8eed-9c671094dad0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:29.670040 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d0337f-5ae6-41ff-8eed-9c671094dad0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:29.673918 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d0337f-5ae6-41ff-8eed-9c671094dad0 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:29.694394 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 928/1879] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:30.240605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242e8f7-b8be-4fb9-b0bd-e7ce793f38be None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:30.242475 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7242e8f7-b8be-4fb9-b0bd-e7ce793f38be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:30.268449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242e8f7-b8be-4fb9-b0bd-e7ce793f38be None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:30.275031 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5689f6b-ea4f-4510-b456-c6a52a5854bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:30.276084 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5689f6b-ea4f-4510-b456-c6a52a5854bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:30.276462 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5689f6b-ea4f-4510-b456-c6a52a5854bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:30.276819 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5689f6b-ea4f-4510-b456-c6a52a5854bd admin admin] PATH_INFO: `/v3/projects/6d55e5af817446ff85e70f08c4bf3e89` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:30.278903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5689f6b-ea4f-4510-b456-c6a52a5854bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d55e5af817446ff85e70f08c4bf3e89)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:30.281503 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5689f6b-ea4f-4510-b456-c6a52a5854bd admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:30.347596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f5689f6b-ea4f-4510-b456-c6a52a5854bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d55e5af817446ff85e70f08c4bf3e89', 'request_id': 'req-f5689f6b-ea4f-4510-b456-c6a52a5854bd'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:30.353862 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f5689f6b-ea4f-4510-b456-c6a52a5854bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d55e5af817446ff85e70f08c4bf3e89', 'request_id': 'req-f5689f6b-ea4f-4510-b456-c6a52a5854bd'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:30.361105 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f5689f6b-ea4f-4510-b456-c6a52a5854bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d55e5af817446ff85e70f08c4bf3e89', 'request_id': 'req-f5689f6b-ea4f-4510-b456-c6a52a5854bd'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:30.385392 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f5689f6b-ea4f-4510-b456-c6a52a5854bd admin admin] The token cache is being invalidate because project 6d55e5af817446ff85e70f08c4bf3e89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:05:30.385681 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f5689f6b-ea4f-4510-b456-c6a52a5854bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:30.386356 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f5689f6b-ea4f-4510-b456-c6a52a5854bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:30.388269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 952/1880] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:05:30 2022] DELETE /identity/v3/projects/6d55e5af817446ff85e70f08c4bf3e89 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:30.392952 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0337f-5ae6-41ff-8eed-9c671094dad0 service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:30.402919 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0337f-5ae6-41ff-8eed-9c671094dad0 service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:30.466501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806b96b-c579-450d-b809-694e9fe2609c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:30.467304 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806b96b-c579-450d-b809-694e9fe2609c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:30.467586 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806b96b-c579-450d-b809-694e9fe2609c admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:30.467857 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806b96b-c579-450d-b809-694e9fe2609c admin admin] PATH_INFO: `/v3/users/4d46ddd3669941ab967a38a15a6eb8a9` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:30.469860 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9806b96b-c579-450d-b809-694e9fe2609c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d46ddd3669941ab967a38a15a6eb8a9)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:30.471864 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9806b96b-c579-450d-b809-694e9fe2609c admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:30.545768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-9806b96b-c579-450d-b809-694e9fe2609c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d46ddd3669941ab967a38a15a6eb8a9'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:30.556341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-9806b96b-c579-450d-b809-694e9fe2609c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d46ddd3669941ab967a38a15a6eb8a9'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:30.557052 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-9806b96b-c579-450d-b809-694e9fe2609c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d46ddd3669941ab967a38a15a6eb8a9'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:30.557742 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-9806b96b-c579-450d-b809-694e9fe2609c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d46ddd3669941ab967a38a15a6eb8a9'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:30.570733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-9806b96b-c579-450d-b809-694e9fe2609c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d46ddd3669941ab967a38a15a6eb8a9'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:30.600975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 929/1881] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:05:30 2022] DELETE /identity/v3/users/4d46ddd3669941ab967a38a15a6eb8a9 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:31.027656 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5689f6b-ea4f-4510-b456-c6a52a5854bd admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:31.029181 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f5689f6b-ea4f-4510-b456-c6a52a5854bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:31.052925 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5689f6b-ea4f-4510-b456-c6a52a5854bd admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:31.155386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992557ee-23bf-4ee6-b9c6-86326a83d807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:31.160030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992557ee-23bf-4ee6-b9c6-86326a83d807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:31.160668 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992557ee-23bf-4ee6-b9c6-86326a83d807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:31.161594 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992557ee-23bf-4ee6-b9c6-86326a83d807 admin admin] PATH_INFO: `/v3/projects/cb33d2a9c51b40818ba8223009114518` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:31.163542 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992557ee-23bf-4ee6-b9c6-86326a83d807 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb33d2a9c51b40818ba8223009114518)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:31.165931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992557ee-23bf-4ee6-b9c6-86326a83d807 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:31.198708 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:31.249477 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-992557ee-23bf-4ee6-b9c6-86326a83d807 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb33d2a9c51b40818ba8223009114518', 'request_id': 'req-992557ee-23bf-4ee6-b9c6-86326a83d807'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:31.267325 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-992557ee-23bf-4ee6-b9c6-86326a83d807 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb33d2a9c51b40818ba8223009114518', 'request_id': 'req-992557ee-23bf-4ee6-b9c6-86326a83d807'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:31.275721 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-992557ee-23bf-4ee6-b9c6-86326a83d807 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb33d2a9c51b40818ba8223009114518', 'request_id': 'req-992557ee-23bf-4ee6-b9c6-86326a83d807'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:31.320641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-992557ee-23bf-4ee6-b9c6-86326a83d807 admin admin] The token cache is being invalidate because project cb33d2a9c51b40818ba8223009114518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:05:31.321120 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-992557ee-23bf-4ee6-b9c6-86326a83d807 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:31.323554 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-992557ee-23bf-4ee6-b9c6-86326a83d807 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:31.331778 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 953/1882] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:05:31 2022] DELETE /identity/v3/projects/cb33d2a9c51b40818ba8223009114518 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:31.336078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd940a-2cd0-4bce-9572-74c04ad7aeaa None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:31.336592 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd940a-2cd0-4bce-9572-74c04ad7aeaa None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:31.337249 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd940a-2cd0-4bce-9572-74c04ad7aeaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:31.430080 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-c7fd940a-2cd0-4bce-9572-74c04ad7aeaa None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:31.432818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-c7fd940a-2cd0-4bce-9572-74c04ad7aeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:31.477723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:31.582737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 930/1883] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:31.591472 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07affb33-bb18-443c-afcd-32f212c58896 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:31.591787 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07affb33-bb18-443c-afcd-32f212c58896 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:31.592043 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07affb33-bb18-443c-afcd-32f212c58896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:31.643113 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-07affb33-bb18-443c-afcd-32f212c58896 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:31.644459 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-07affb33-bb18-443c-afcd-32f212c58896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:31.668018 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 954/1884] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:31.674950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd940a-2cd0-4bce-9572-74c04ad7aeaa None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:31.692321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd940a-2cd0-4bce-9572-74c04ad7aeaa None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:31.697904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3b766-3f25-4bb8-be54-108e84ab5dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:31.699054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3b766-3f25-4bb8-be54-108e84ab5dad admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:31.699346 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3b766-3f25-4bb8-be54-108e84ab5dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:31.699617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3b766-3f25-4bb8-be54-108e84ab5dad admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:31.700358 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d3b766-3f25-4bb8-be54-108e84ab5dad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:31.700665 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d3b766-3f25-4bb8-be54-108e84ab5dad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:31.709133 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d3b766-3f25-4bb8-be54-108e84ab5dad admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:31.722068 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 931/1885] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:31.727495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07affb33-bb18-443c-afcd-32f212c58896 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:31.741741 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07affb33-bb18-443c-afcd-32f212c58896 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:31.748110 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e02ba-7ccd-40ff-9a7b-85c476a54d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:31.749002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e02ba-7ccd-40ff-9a7b-85c476a54d53 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:31.749002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e02ba-7ccd-40ff-9a7b-85c476a54d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:31.749445 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e02ba-7ccd-40ff-9a7b-85c476a54d53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:31.751300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e02ba-7ccd-40ff-9a7b-85c476a54d53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:31.753510 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e02ba-7ccd-40ff-9a7b-85c476a54d53 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:31.787583 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 955/1886] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:05:31 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:05:31.792789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3b766-3f25-4bb8-be54-108e84ab5dad admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:31.811041 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3b766-3f25-4bb8-be54-108e84ab5dad admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:31.916909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b30c5-604c-4682-8139-7dc026beb226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:31.917798 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b30c5-604c-4682-8139-7dc026beb226 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:31.918574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b30c5-604c-4682-8139-7dc026beb226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:31.919414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b30c5-604c-4682-8139-7dc026beb226 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:31.922622 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b30c5-604c-4682-8139-7dc026beb226 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:31.925327 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b30c5-604c-4682-8139-7dc026beb226 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:31.954466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 932/1887] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:05:31 2022] POST /identity/v3/users => generated 287 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:05:31.961797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e02ba-7ccd-40ff-9a7b-85c476a54d53 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:31.979002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e02ba-7ccd-40ff-9a7b-85c476a54d53 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:31.987900 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039b0cb-fbba-4145-8839-ff2fe0ddea5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:31.988637 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039b0cb-fbba-4145-8839-ff2fe0ddea5d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:31.988923 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039b0cb-fbba-4145-8839-ff2fe0ddea5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:31.989205 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039b0cb-fbba-4145-8839-ff2fe0ddea5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:31.989944 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039b0cb-fbba-4145-8839-ff2fe0ddea5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:31.992032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039b0cb-fbba-4145-8839-ff2fe0ddea5d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:32.018266 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 956/1888] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:32.023584 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b30c5-604c-4682-8139-7dc026beb226 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:32.047086 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b30c5-604c-4682-8139-7dc026beb226 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:32.053495 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cb6ce-c022-4f08-babd-c37f04fae99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:32.054250 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cb6ce-c022-4f08-babd-c37f04fae99b admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:32.054685 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cb6ce-c022-4f08-babd-c37f04fae99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:32.055152 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cb6ce-c022-4f08-babd-c37f04fae99b admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:32.056040 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025cb6ce-c022-4f08-babd-c37f04fae99b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:32.058345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025cb6ce-c022-4f08-babd-c37f04fae99b admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:32.073544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 933/1889] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:32.074900 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039b0cb-fbba-4145-8839-ff2fe0ddea5d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:32.088590 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039b0cb-fbba-4145-8839-ff2fe0ddea5d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:32.096080 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f0890-45fb-4d19-81f8-9807c0f2dbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:32.099077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f0890-45fb-4d19-81f8-9807c0f2dbd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:32.099949 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f0890-45fb-4d19-81f8-9807c0f2dbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:32.100675 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f0890-45fb-4d19-81f8-9807c0f2dbd8 admin admin] PATH_INFO: `/v3/projects/563df53d3d894b6db3c703a076b9710a/users/fa4c392318044e78bf8ccd03ce203bef/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:32.162361 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f0890-45fb-4d19-81f8-9807c0f2dbd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=563df53d3d894b6db3c703a076b9710a, user_id=fa4c392318044e78bf8ccd03ce203bef, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:32.166352 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f0890-45fb-4d19-81f8-9807c0f2dbd8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:32.192461 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 957/1890] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:32 2022] PUT /identity/v3/projects/563df53d3d894b6db3c703a076b9710a/users/fa4c392318044e78bf8ccd03ce203bef/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:32.200079 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f20ca-3628-4bef-beaf-f9111dff353f None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:32.200915 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f20ca-3628-4bef-beaf-f9111dff353f None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:32.201243 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f20ca-3628-4bef-beaf-f9111dff353f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:32.231436 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-e45f20ca-3628-4bef-beaf-f9111dff353f None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:32.232977 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-e45f20ca-3628-4bef-beaf-f9111dff353f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:32.365025 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 934/1891] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:32.387254 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f0890-45fb-4d19-81f8-9807c0f2dbd8 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:32.389944 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-771f0890-45fb-4d19-81f8-9807c0f2dbd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:32.587977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f0890-45fb-4d19-81f8-9807c0f2dbd8 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:32.598563 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506ba49-770f-4701-873e-18893cf25b98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:32.599949 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506ba49-770f-4701-873e-18893cf25b98 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:32.600364 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506ba49-770f-4701-873e-18893cf25b98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:32.600652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506ba49-770f-4701-873e-18893cf25b98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:32.615279 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c506ba49-770f-4701-873e-18893cf25b98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:32.616421 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c506ba49-770f-4701-873e-18893cf25b98 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:32.632027 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 958/1892] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:33.869257 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918dce2-d165-4bc9-a0a5-0df068bdf7f3 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:33.869823 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918dce2-d165-4bc9-a0a5-0df068bdf7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:33.869823 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918dce2-d165-4bc9-a0a5-0df068bdf7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:33.929877 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-7918dce2-d165-4bc9-a0a5-0df068bdf7f3 None None] MFA Rules not processed for user `fa4c392318044e78bf8ccd03ce203bef`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:33.931204 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-7918dce2-d165-4bc9-a0a5-0df068bdf7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:33.984261 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:34.069877 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 935/1893] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:34.075219 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506ba49-770f-4701-873e-18893cf25b98 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:34.094373 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506ba49-770f-4701-873e-18893cf25b98 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:34.106930 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024cb68-f2cf-4ab1-9615-4e8fc9a9922c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:34.108293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024cb68-f2cf-4ab1-9615-4e8fc9a9922c admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:34.108846 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024cb68-f2cf-4ab1-9615-4e8fc9a9922c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:34.109589 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024cb68-f2cf-4ab1-9615-4e8fc9a9922c admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:34.110658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024cb68-f2cf-4ab1-9615-4e8fc9a9922c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:34.113775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024cb68-f2cf-4ab1-9615-4e8fc9a9922c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:34.196227 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 959/1894] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:05:34 2022] POST /identity/v3/projects => generated 371 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:34.200575 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918dce2-d165-4bc9-a0a5-0df068bdf7f3 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:34.220655 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918dce2-d165-4bc9-a0a5-0df068bdf7f3 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:34.383646 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7063a2a-1c9d-4303-8007-9763c7e74e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:34.385929 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7063a2a-1c9d-4303-8007-9763c7e74e48 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:34.386218 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7063a2a-1c9d-4303-8007-9763c7e74e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:34.386526 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7063a2a-1c9d-4303-8007-9763c7e74e48 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:34.387702 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7063a2a-1c9d-4303-8007-9763c7e74e48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:34.393922 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7063a2a-1c9d-4303-8007-9763c7e74e48 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:34.438213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 936/1895] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:05:34 2022] POST /identity/v3/users => generated 292 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:05:34.442774 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024cb68-f2cf-4ab1-9615-4e8fc9a9922c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:34.455527 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024cb68-f2cf-4ab1-9615-4e8fc9a9922c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:34.471829 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8468df-0ae4-4d1c-9316-7a85245159da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:34.473059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8468df-0ae4-4d1c-9316-7a85245159da admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:34.473655 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8468df-0ae4-4d1c-9316-7a85245159da admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:34.474215 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8468df-0ae4-4d1c-9316-7a85245159da admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:34.475284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8468df-0ae4-4d1c-9316-7a85245159da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:34.478428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8468df-0ae4-4d1c-9316-7a85245159da admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:34.503613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7063a2a-1c9d-4303-8007-9763c7e74e48 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:34.506371 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 960/1896] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:34.516218 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7063a2a-1c9d-4303-8007-9763c7e74e48 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:34.532756 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768fb43-81ef-4037-ab63-2dc487fe17a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:34.534494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768fb43-81ef-4037-ab63-2dc487fe17a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:34.534770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768fb43-81ef-4037-ab63-2dc487fe17a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:34.535070 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768fb43-81ef-4037-ab63-2dc487fe17a3 admin admin] PATH_INFO: `/v3/projects/21903e884eed4f2cb0618ac7b2a2c9d4/users/c3f20ea085e34f748a8aee94300351cd/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:34.559961 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a768fb43-81ef-4037-ab63-2dc487fe17a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21903e884eed4f2cb0618ac7b2a2c9d4, user_id=c3f20ea085e34f748a8aee94300351cd, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:34.563009 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a768fb43-81ef-4037-ab63-2dc487fe17a3 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:34.577738 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 937/1897] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:34 2022] PUT /identity/v3/projects/21903e884eed4f2cb0618ac7b2a2c9d4/users/c3f20ea085e34f748a8aee94300351cd/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:34.581704 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8468df-0ae4-4d1c-9316-7a85245159da admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:34.597343 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8468df-0ae4-4d1c-9316-7a85245159da admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:34.611608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:34.688547 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ec527-efcc-426e-867c-be5ecfd97ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:34.689265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ec527-efcc-426e-867c-be5ecfd97ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:34.689551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ec527-efcc-426e-867c-be5ecfd97ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:34.689828 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ec527-efcc-426e-867c-be5ecfd97ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:34.690597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ec527-efcc-426e-867c-be5ecfd97ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:34.692219 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ec527-efcc-426e-867c-be5ecfd97ea3 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:34.704391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 961/1898] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:34.707935 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768fb43-81ef-4037-ab63-2dc487fe17a3 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:34.719767 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768fb43-81ef-4037-ab63-2dc487fe17a3 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:34.724892 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9506b39-9da5-42ca-a8e4-f1e5158e533e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:34.725817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9506b39-9da5-42ca-a8e4-f1e5158e533e admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:34.726104 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9506b39-9da5-42ca-a8e4-f1e5158e533e admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:34.726455 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9506b39-9da5-42ca-a8e4-f1e5158e533e admin admin] PATH_INFO: `/v3/projects/21903e884eed4f2cb0618ac7b2a2c9d4/users/c3f20ea085e34f748a8aee94300351cd/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:34.728678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9506b39-9da5-42ca-a8e4-f1e5158e533e admin admin] RBAC: Authorizing `identity:create_grant(project_id=21903e884eed4f2cb0618ac7b2a2c9d4, user_id=c3f20ea085e34f748a8aee94300351cd, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:34.731506 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9506b39-9da5-42ca-a8e4-f1e5158e533e admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:34.743043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 938/1899] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:34 2022] PUT /identity/v3/projects/21903e884eed4f2cb0618ac7b2a2c9d4/users/c3f20ea085e34f748a8aee94300351cd/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:35.928833 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d9280-e249-405a-8282-8e31fa6d4a8c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:35.929386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d9280-e249-405a-8282-8e31fa6d4a8c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:35.929770 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d9280-e249-405a-8282-8e31fa6d4a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:35.982414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-040d9280-e249-405a-8282-8e31fa6d4a8c None None] MFA Rules not processed for user `c3f20ea085e34f748a8aee94300351cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:35.985309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-040d9280-e249-405a-8282-8e31fa6d4a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:36.104254 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 962/1900] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:35 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:36.123818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9506b39-9da5-42ca-a8e4-f1e5158e533e admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:36.128088 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f9506b39-9da5-42ca-a8e4-f1e5158e533e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:36.220978 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9506b39-9da5-42ca-a8e4-f1e5158e533e admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:36.225086 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a5df9-69c5-4558-9eb5-9b89ca09e568 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:36.225771 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a5df9-69c5-4558-9eb5-9b89ca09e568 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:36.226056 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a5df9-69c5-4558-9eb5-9b89ca09e568 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:36.226367 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a5df9-69c5-4558-9eb5-9b89ca09e568 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:36.265935 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:36.317732 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a5df9-69c5-4558-9eb5-9b89ca09e568 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:36.319592 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a5df9-69c5-4558-9eb5-9b89ca09e568 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:36.336858 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 939/1901] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:38.251721 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d9280-e249-405a-8282-8e31fa6d4a8c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:38.268860 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d9280-e249-405a-8282-8e31fa6d4a8c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:38.276982 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee56c9-d48d-4aba-b0d6-262eaf01b2d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:38.278431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee56c9-d48d-4aba-b0d6-262eaf01b2d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:38.279133 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee56c9-d48d-4aba-b0d6-262eaf01b2d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:38.279774 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee56c9-d48d-4aba-b0d6-262eaf01b2d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:38.306388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee56c9-d48d-4aba-b0d6-262eaf01b2d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:38.309544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee56c9-d48d-4aba-b0d6-262eaf01b2d4 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:38.327631 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 963/1902] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:38 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:42.095999 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a5df9-69c5-4558-9eb5-9b89ca09e568 service cinder] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:42.097555 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-509a5df9-69c5-4558-9eb5-9b89ca09e568 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:42.188749 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a5df9-69c5-4558-9eb5-9b89ca09e568 service cinder] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:42.194098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f599ee-fc33-4145-a5d2-646dee9656a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:42.195100 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f599ee-fc33-4145-a5d2-646dee9656a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:42.195385 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f599ee-fc33-4145-a5d2-646dee9656a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:42.195659 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f599ee-fc33-4145-a5d2-646dee9656a8 admin admin] PATH_INFO: `/v3/users/8b9579d5fe3f4bb5a7fa432013f304e6` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:42.196872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f599ee-fc33-4145-a5d2-646dee9656a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b9579d5fe3f4bb5a7fa432013f304e6)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:42.198972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f599ee-fc33-4145-a5d2-646dee9656a8 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:42.284121 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-a7f599ee-fc33-4145-a5d2-646dee9656a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b9579d5fe3f4bb5a7fa432013f304e6'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:42.295456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-a7f599ee-fc33-4145-a5d2-646dee9656a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b9579d5fe3f4bb5a7fa432013f304e6'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:42.296142 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-a7f599ee-fc33-4145-a5d2-646dee9656a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b9579d5fe3f4bb5a7fa432013f304e6'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:42.297989 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-a7f599ee-fc33-4145-a5d2-646dee9656a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b9579d5fe3f4bb5a7fa432013f304e6'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:42.307274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-a7f599ee-fc33-4145-a5d2-646dee9656a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b9579d5fe3f4bb5a7fa432013f304e6'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:42.342263 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 940/1903] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:05:42 2022] DELETE /identity/v3/users/8b9579d5fe3f4bb5a7fa432013f304e6 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:42.349802 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e3cb4-df5e-4a88-968f-ab775574beaa None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:42.350424 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e3cb4-df5e-4a88-968f-ab775574beaa None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:42.350870 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e3cb4-df5e-4a88-968f-ab775574beaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:42.379699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d31e3cb4-df5e-4a88-968f-ab775574beaa None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:42.381375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d31e3cb4-df5e-4a88-968f-ab775574beaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:42.461807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 964/1904] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:42.502158 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f599ee-fc33-4145-a5d2-646dee9656a8 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:42.504096 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-a7f599ee-fc33-4145-a5d2-646dee9656a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:42.601740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:42.607974 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f599ee-fc33-4145-a5d2-646dee9656a8 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:42.618893 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bef380-9bb3-4646-a12c-d2b8be10fc06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:42.621863 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bef380-9bb3-4646-a12c-d2b8be10fc06 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:42.623536 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bef380-9bb3-4646-a12c-d2b8be10fc06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:42.624169 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bef380-9bb3-4646-a12c-d2b8be10fc06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:42.639158 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bef380-9bb3-4646-a12c-d2b8be10fc06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:42.641768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bef380-9bb3-4646-a12c-d2b8be10fc06 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:42.659235 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 941/1905] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:05:43.176742 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e3cb4-df5e-4a88-968f-ab775574beaa None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:43.182955 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d31e3cb4-df5e-4a88-968f-ab775574beaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:43.212927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e3cb4-df5e-4a88-968f-ab775574beaa None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:43.224340 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d58a59-43fe-430d-9ad1-7fa2befbda9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:43.225589 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d58a59-43fe-430d-9ad1-7fa2befbda9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:43.226161 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d58a59-43fe-430d-9ad1-7fa2befbda9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:43.226869 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d58a59-43fe-430d-9ad1-7fa2befbda9e admin admin] PATH_INFO: `/v3/projects/dbb8c80a56f544c9a0203db42382f72f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:43.229014 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d58a59-43fe-430d-9ad1-7fa2befbda9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbb8c80a56f544c9a0203db42382f72f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:43.231557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d58a59-43fe-430d-9ad1-7fa2befbda9e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:43.259734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:43.311669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-16d58a59-43fe-430d-9ad1-7fa2befbda9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbb8c80a56f544c9a0203db42382f72f', 'request_id': 'req-16d58a59-43fe-430d-9ad1-7fa2befbda9e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:43.320341 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-16d58a59-43fe-430d-9ad1-7fa2befbda9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbb8c80a56f544c9a0203db42382f72f', 'request_id': 'req-16d58a59-43fe-430d-9ad1-7fa2befbda9e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:43.330423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-16d58a59-43fe-430d-9ad1-7fa2befbda9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbb8c80a56f544c9a0203db42382f72f', 'request_id': 'req-16d58a59-43fe-430d-9ad1-7fa2befbda9e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:43.351188 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-16d58a59-43fe-430d-9ad1-7fa2befbda9e admin admin] The token cache is being invalidate because project dbb8c80a56f544c9a0203db42382f72f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:05:43.352321 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-16d58a59-43fe-430d-9ad1-7fa2befbda9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:43.353072 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-16d58a59-43fe-430d-9ad1-7fa2befbda9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:43.359896 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 965/1906] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:05:43 2022] DELETE /identity/v3/projects/dbb8c80a56f544c9a0203db42382f72f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:43.363403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bef380-9bb3-4646-a12c-d2b8be10fc06 service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:43.374381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bef380-9bb3-4646-a12c-d2b8be10fc06 service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:43.466875 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862f809-65ce-423c-ae57-2f296e88a78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:43.469364 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862f809-65ce-423c-ae57-2f296e88a78c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:43.470264 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862f809-65ce-423c-ae57-2f296e88a78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:43.470626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862f809-65ce-423c-ae57-2f296e88a78c admin admin] PATH_INFO: `/v3/users/a139ea014fc344e2a1f2ecfb6aa6fa51` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:43.473540 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862f809-65ce-423c-ae57-2f296e88a78c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a139ea014fc344e2a1f2ecfb6aa6fa51)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:43.479738 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862f809-65ce-423c-ae57-2f296e88a78c admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:43.584607 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-3862f809-65ce-423c-ae57-2f296e88a78c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a139ea014fc344e2a1f2ecfb6aa6fa51'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:43.595713 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-3862f809-65ce-423c-ae57-2f296e88a78c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a139ea014fc344e2a1f2ecfb6aa6fa51'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:43.596514 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-3862f809-65ce-423c-ae57-2f296e88a78c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a139ea014fc344e2a1f2ecfb6aa6fa51'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:43.597357 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-3862f809-65ce-423c-ae57-2f296e88a78c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a139ea014fc344e2a1f2ecfb6aa6fa51'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:43.607978 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-3862f809-65ce-423c-ae57-2f296e88a78c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a139ea014fc344e2a1f2ecfb6aa6fa51'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:43.635554 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 942/1907] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:05:43 2022] DELETE /identity/v3/users/a139ea014fc344e2a1f2ecfb6aa6fa51 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:44.079845 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d58a59-43fe-430d-9ad1-7fa2befbda9e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:44.081569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-16d58a59-43fe-430d-9ad1-7fa2befbda9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:44.103346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d58a59-43fe-430d-9ad1-7fa2befbda9e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:44.236048 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0e62b-45fe-4a2b-a798-5d1b594d3078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:44.238136 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0e62b-45fe-4a2b-a798-5d1b594d3078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:44.238942 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0e62b-45fe-4a2b-a798-5d1b594d3078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:44.239615 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0e62b-45fe-4a2b-a798-5d1b594d3078 admin admin] PATH_INFO: `/v3/projects/26130a4c94624fa1af487cee289237a8` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:44.242241 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0e62b-45fe-4a2b-a798-5d1b594d3078 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26130a4c94624fa1af487cee289237a8)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:44.246766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0e62b-45fe-4a2b-a798-5d1b594d3078 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:44.363114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-74e0e62b-45fe-4a2b-a798-5d1b594d3078 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26130a4c94624fa1af487cee289237a8', 'request_id': 'req-74e0e62b-45fe-4a2b-a798-5d1b594d3078'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:44.370228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-74e0e62b-45fe-4a2b-a798-5d1b594d3078 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26130a4c94624fa1af487cee289237a8', 'request_id': 'req-74e0e62b-45fe-4a2b-a798-5d1b594d3078'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:44.378790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-74e0e62b-45fe-4a2b-a798-5d1b594d3078 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26130a4c94624fa1af487cee289237a8', 'request_id': 'req-74e0e62b-45fe-4a2b-a798-5d1b594d3078'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:44.394391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-74e0e62b-45fe-4a2b-a798-5d1b594d3078 admin admin] The token cache is being invalidate because project 26130a4c94624fa1af487cee289237a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:05:44.394666 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-74e0e62b-45fe-4a2b-a798-5d1b594d3078 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:44.395385 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-74e0e62b-45fe-4a2b-a798-5d1b594d3078 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:44.397008 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 966/1908] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:05:44 2022] DELETE /identity/v3/projects/26130a4c94624fa1af487cee289237a8 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:05:44.402175 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ef2b4-13ef-4619-a2c5-221afc844c44 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:44.403042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ef2b4-13ef-4619-a2c5-221afc844c44 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:44.403388 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ef2b4-13ef-4619-a2c5-221afc844c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:44.430385 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-704ef2b4-13ef-4619-a2c5-221afc844c44 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:44.430385 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-704ef2b4-13ef-4619-a2c5-221afc844c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:44.533258 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 943/1909] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:44.545733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c198a5e-d108-4763-9a63-ea8323116069 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:44.546750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c198a5e-d108-4763-9a63-ea8323116069 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:44.547138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c198a5e-d108-4763-9a63-ea8323116069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:44.571701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4c198a5e-d108-4763-9a63-ea8323116069 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:44.574105 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4c198a5e-d108-4763-9a63-ea8323116069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:44.588227 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 967/1910] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:05:44.592733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ef2b4-13ef-4619-a2c5-221afc844c44 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:44.605358 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ef2b4-13ef-4619-a2c5-221afc844c44 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:44.611777 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2a541-9c3c-42e3-bec4-197e5a93bb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:44.612461 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2a541-9c3c-42e3-bec4-197e5a93bb07 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:44.612743 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2a541-9c3c-42e3-bec4-197e5a93bb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:44.613028 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2a541-9c3c-42e3-bec4-197e5a93bb07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:44.613844 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2a541-9c3c-42e3-bec4-197e5a93bb07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:44.614056 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2a541-9c3c-42e3-bec4-197e5a93bb07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:44.615818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2a541-9c3c-42e3-bec4-197e5a93bb07 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:44.624738 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:44.632816 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 944/1911] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:44.636263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c198a5e-d108-4763-9a63-ea8323116069 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:44.647689 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c198a5e-d108-4763-9a63-ea8323116069 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:44.653134 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e9879-c28f-4426-b15a-d4ba8235ffee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:44.653818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e9879-c28f-4426-b15a-d4ba8235ffee admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:44.654167 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e9879-c28f-4426-b15a-d4ba8235ffee admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:44.654443 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e9879-c28f-4426-b15a-d4ba8235ffee admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:44.655393 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e9879-c28f-4426-b15a-d4ba8235ffee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:44.658299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e9879-c28f-4426-b15a-d4ba8235ffee admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:44.691323 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 968/1912] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:05:44 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:44.693044 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2a541-9c3c-42e3-bec4-197e5a93bb07 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:44.704728 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2a541-9c3c-42e3-bec4-197e5a93bb07 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:44.781934 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3f021-521b-4876-915e-76e4d3c06b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:44.782808 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3f021-521b-4876-915e-76e4d3c06b49 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:44.783077 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3f021-521b-4876-915e-76e4d3c06b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:44.783357 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3f021-521b-4876-915e-76e4d3c06b49 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:44.784072 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3f021-521b-4876-915e-76e4d3c06b49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:44.786059 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3f021-521b-4876-915e-76e4d3c06b49 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:44.819129 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 945/1913] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:05:44 2022] POST /identity/v3/users => generated 291 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:44.823454 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e9879-c28f-4426-b15a-d4ba8235ffee admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:44.835460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e9879-c28f-4426-b15a-d4ba8235ffee admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:44.840426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550bee6-501f-4448-9c15-0da716a62f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:44.841133 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550bee6-501f-4448-9c15-0da716a62f84 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:44.841415 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550bee6-501f-4448-9c15-0da716a62f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:44.841691 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550bee6-501f-4448-9c15-0da716a62f84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:44.842504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b550bee6-501f-4448-9c15-0da716a62f84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:44.844361 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b550bee6-501f-4448-9c15-0da716a62f84 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:44.846085 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:44.864446 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3f021-521b-4876-915e-76e4d3c06b49 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:44.865189 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 969/1914] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:44.877723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3f021-521b-4876-915e-76e4d3c06b49 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:44.885077 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47f6b9-1852-44b4-9858-998b4e1b7dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:44.885905 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47f6b9-1852-44b4-9858-998b4e1b7dbe admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:44.886267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47f6b9-1852-44b4-9858-998b4e1b7dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:44.886580 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47f6b9-1852-44b4-9858-998b4e1b7dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:44.887615 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47f6b9-1852-44b4-9858-998b4e1b7dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:44.889512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47f6b9-1852-44b4-9858-998b4e1b7dbe admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:44.902037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 946/1915] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:05:44.906977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550bee6-501f-4448-9c15-0da716a62f84 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:44.917532 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550bee6-501f-4448-9c15-0da716a62f84 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:44.923216 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca320a7-3998-4cc8-987c-ddd9153336f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:44.924144 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca320a7-3998-4cc8-987c-ddd9153336f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:44.924433 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca320a7-3998-4cc8-987c-ddd9153336f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:44.924712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca320a7-3998-4cc8-987c-ddd9153336f0 admin admin] PATH_INFO: `/v3/projects/369bcd27406d43ed930e94e255ff8dc1/users/173a39e8e22e478ababe1341fb119b73/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:44.946384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca320a7-3998-4cc8-987c-ddd9153336f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=369bcd27406d43ed930e94e255ff8dc1, user_id=173a39e8e22e478ababe1341fb119b73, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:44.948701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca320a7-3998-4cc8-987c-ddd9153336f0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:44.964078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 970/1916] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:44 2022] PUT /identity/v3/projects/369bcd27406d43ed930e94e255ff8dc1/users/173a39e8e22e478ababe1341fb119b73/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:44.977011 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8e91d-4584-45d4-b264-a254be316ed3 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:44.977866 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8e91d-4584-45d4-b264-a254be316ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:44.978045 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8e91d-4584-45d4-b264-a254be316ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:45.014262 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-55c8e91d-4584-45d4-b264-a254be316ed3 None None] MFA Rules not processed for user `173a39e8e22e478ababe1341fb119b73`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:45.015341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-55c8e91d-4584-45d4-b264-a254be316ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:45.102727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 947/1917] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:45.109568 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca320a7-3998-4cc8-987c-ddd9153336f0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:45.191820 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca320a7-3998-4cc8-987c-ddd9153336f0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:45.197247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f4126-723a-4fea-9ede-1323a52b78d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:45.198042 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f4126-723a-4fea-9ede-1323a52b78d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:45.198452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f4126-723a-4fea-9ede-1323a52b78d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:45.198831 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f4126-723a-4fea-9ede-1323a52b78d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:45.199717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f4126-723a-4fea-9ede-1323a52b78d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:45.201772 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f4126-723a-4fea-9ede-1323a52b78d7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:45.232138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 971/1918] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:05:45 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:05:45.237111 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8e91d-4584-45d4-b264-a254be316ed3 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:45.249016 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8e91d-4584-45d4-b264-a254be316ed3 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:45.342889 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c3230-e939-460d-b830-b15dddbaf5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:45.344059 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c3230-e939-460d-b830-b15dddbaf5e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:45.344638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c3230-e939-460d-b830-b15dddbaf5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:45.345053 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c3230-e939-460d-b830-b15dddbaf5e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:45.346303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c3230-e939-460d-b830-b15dddbaf5e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:45.350036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c3230-e939-460d-b830-b15dddbaf5e4 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:45.367165 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:45.386916 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 948/1919] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:05:45 2022] POST /identity/v3/users => generated 299 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:45.391768 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f4126-723a-4fea-9ede-1323a52b78d7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:45.403193 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f4126-723a-4fea-9ede-1323a52b78d7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:45.409156 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f3492-a2bf-41f8-8374-d618e2fda46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:45.409928 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f3492-a2bf-41f8-8374-d618e2fda46a admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:45.410328 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f3492-a2bf-41f8-8374-d618e2fda46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:45.410777 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f3492-a2bf-41f8-8374-d618e2fda46a admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:45.411948 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f3492-a2bf-41f8-8374-d618e2fda46a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:45.415072 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f3492-a2bf-41f8-8374-d618e2fda46a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:45.434688 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 972/1920] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:05:45.440370 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c3230-e939-460d-b830-b15dddbaf5e4 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:45.453746 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c3230-e939-460d-b830-b15dddbaf5e4 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:45.460435 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c360e7d-9497-4460-9933-c016d4df97c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:45.461467 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c360e7d-9497-4460-9933-c016d4df97c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:45.461842 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c360e7d-9497-4460-9933-c016d4df97c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:45.462281 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c360e7d-9497-4460-9933-c016d4df97c3 admin admin] PATH_INFO: `/v3/projects/20aa79df21104b3994c5f658401c9446/users/b9cb4a9a3de342ad8941e1b2a00c03dd/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:45.482345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c360e7d-9497-4460-9933-c016d4df97c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20aa79df21104b3994c5f658401c9446, user_id=b9cb4a9a3de342ad8941e1b2a00c03dd, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:45.486122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c360e7d-9497-4460-9933-c016d4df97c3 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:45.508149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 949/1921] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:45 2022] PUT /identity/v3/projects/20aa79df21104b3994c5f658401c9446/users/b9cb4a9a3de342ad8941e1b2a00c03dd/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:05:45.513273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f3492-a2bf-41f8-8374-d618e2fda46a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:45.527398 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f3492-a2bf-41f8-8374-d618e2fda46a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:45.604397 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8ab5f-a723-4747-9471-3f5040a6ad84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:45.605560 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8ab5f-a723-4747-9471-3f5040a6ad84 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:45.606058 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8ab5f-a723-4747-9471-3f5040a6ad84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:45.606485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8ab5f-a723-4747-9471-3f5040a6ad84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:45.607384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8ab5f-a723-4747-9471-3f5040a6ad84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:45.610131 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8ab5f-a723-4747-9471-3f5040a6ad84 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:45.621434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 973/1922] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:05:45.626380 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c360e7d-9497-4460-9933-c016d4df97c3 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:45.637442 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c360e7d-9497-4460-9933-c016d4df97c3 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:45.644352 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9882b4-1eab-44b7-bd13-ed731cab6a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:45.645344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9882b4-1eab-44b7-bd13-ed731cab6a8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:45.645681 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9882b4-1eab-44b7-bd13-ed731cab6a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:45.646021 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9882b4-1eab-44b7-bd13-ed731cab6a8e admin admin] PATH_INFO: `/v3/projects/20aa79df21104b3994c5f658401c9446/users/b9cb4a9a3de342ad8941e1b2a00c03dd/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:45.648339 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9882b4-1eab-44b7-bd13-ed731cab6a8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=20aa79df21104b3994c5f658401c9446, user_id=b9cb4a9a3de342ad8941e1b2a00c03dd, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:45.651086 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9882b4-1eab-44b7-bd13-ed731cab6a8e admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:45.664743 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 950/1923] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:45 2022] PUT /identity/v3/projects/20aa79df21104b3994c5f658401c9446/users/b9cb4a9a3de342ad8941e1b2a00c03dd/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:45.676322 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644a46f-e8a2-417f-a043-ff9e40e192d1 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:45.676767 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644a46f-e8a2-417f-a043-ff9e40e192d1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:45.677182 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644a46f-e8a2-417f-a043-ff9e40e192d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:45.709299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c644a46f-e8a2-417f-a043-ff9e40e192d1 None None] MFA Rules not processed for user `b9cb4a9a3de342ad8941e1b2a00c03dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:45.710831 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c644a46f-e8a2-417f-a043-ff9e40e192d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:45.745688 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:45.792341 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 974/1924] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:45 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:45.803726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9882b4-1eab-44b7-bd13-ed731cab6a8e admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:45.805501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-3e9882b4-1eab-44b7-bd13-ed731cab6a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:45.900370 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9882b4-1eab-44b7-bd13-ed731cab6a8e admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:45.904065 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c7d3d-f2b5-4c4a-bca2-91d6de04ae40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:45.904749 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c7d3d-f2b5-4c4a-bca2-91d6de04ae40 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:45.905291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c7d3d-f2b5-4c4a-bca2-91d6de04ae40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:45.905568 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c7d3d-f2b5-4c4a-bca2-91d6de04ae40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:45.985275 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c7d3d-f2b5-4c4a-bca2-91d6de04ae40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:45.987012 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c7d3d-f2b5-4c4a-bca2-91d6de04ae40 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:45.997421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 951/1925] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:51.563931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644a46f-e8a2-417f-a043-ff9e40e192d1 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:51.574973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644a46f-e8a2-417f-a043-ff9e40e192d1 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:51.579702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ecf02-d3f1-4464-a091-98d8660e99be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:51.580377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ecf02-d3f1-4464-a091-98d8660e99be service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:51.580659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ecf02-d3f1-4464-a091-98d8660e99be service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:51.580972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ecf02-d3f1-4464-a091-98d8660e99be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:51.600654 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ecf02-d3f1-4464-a091-98d8660e99be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:51.603676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ecf02-d3f1-4464-a091-98d8660e99be service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:51.620273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 975/1926] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:55.992567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=65,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:56.571463 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c7d3d-f2b5-4c4a-bca2-91d6de04ae40 service cinder] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:56.573630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-b32c7d3d-f2b5-4c4a-bca2-91d6de04ae40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:56.681096 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c7d3d-f2b5-4c4a-bca2-91d6de04ae40 service cinder] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:56.690859 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d9de0-b102-48f5-b312-d6a1b801c10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:56.691776 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d9de0-b102-48f5-b312-d6a1b801c10e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:56.692347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d9de0-b102-48f5-b312-d6a1b801c10e admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:56.692771 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d9de0-b102-48f5-b312-d6a1b801c10e admin admin] PATH_INFO: `/v3/users/4c6297456efe476e9a656e1a52f361d4` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:56.694339 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d9de0-b102-48f5-b312-d6a1b801c10e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c6297456efe476e9a656e1a52f361d4)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:56.697689 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d9de0-b102-48f5-b312-d6a1b801c10e admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:56.764996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-780d9de0-b102-48f5-b312-d6a1b801c10e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c6297456efe476e9a656e1a52f361d4'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:56.773955 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-780d9de0-b102-48f5-b312-d6a1b801c10e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c6297456efe476e9a656e1a52f361d4'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:56.774690 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-780d9de0-b102-48f5-b312-d6a1b801c10e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c6297456efe476e9a656e1a52f361d4'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:56.775464 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-780d9de0-b102-48f5-b312-d6a1b801c10e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c6297456efe476e9a656e1a52f361d4'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:56.786107 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-780d9de0-b102-48f5-b312-d6a1b801c10e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c6297456efe476e9a656e1a52f361d4'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:56.809693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 952/1927] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:05:56 2022] DELETE /identity/v3/users/4c6297456efe476e9a656e1a52f361d4 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:56.815166 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa0e16-c3f9-4ca5-8da0-732d469399e5 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:56.815663 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa0e16-c3f9-4ca5-8da0-732d469399e5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:56.815855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa0e16-c3f9-4ca5-8da0-732d469399e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:56.834114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1faa0e16-c3f9-4ca5-8da0-732d469399e5 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:56.835394 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1faa0e16-c3f9-4ca5-8da0-732d469399e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:56.906645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 976/1928] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:56.916866 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d9de0-b102-48f5-b312-d6a1b801c10e admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:56.918203 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-780d9de0-b102-48f5-b312-d6a1b801c10e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:57.007231 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d9de0-b102-48f5-b312-d6a1b801c10e admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:57.013336 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb6d28-d314-4ca8-bd7d-69bc0e3f7293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:57.014020 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb6d28-d314-4ca8-bd7d-69bc0e3f7293 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:57.014355 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb6d28-d314-4ca8-bd7d-69bc0e3f7293 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:57.014625 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb6d28-d314-4ca8-bd7d-69bc0e3f7293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:57.026047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb6d28-d314-4ca8-bd7d-69bc0e3f7293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:57.029090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb6d28-d314-4ca8-bd7d-69bc0e3f7293 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:57.041620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 953/1929] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:05:57.573439 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa0e16-c3f9-4ca5-8da0-732d469399e5 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:57.575904 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1faa0e16-c3f9-4ca5-8da0-732d469399e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:57.600879 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa0e16-c3f9-4ca5-8da0-732d469399e5 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:57.606028 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320615f2-d309-4730-8d17-244c0a8228ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:57.607018 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320615f2-d309-4730-8d17-244c0a8228ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:57.607306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320615f2-d309-4730-8d17-244c0a8228ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:57.607587 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320615f2-d309-4730-8d17-244c0a8228ce admin admin] PATH_INFO: `/v3/projects/8cbe6d45be104dfaacc3afd241b940ca` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:57.617073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320615f2-d309-4730-8d17-244c0a8228ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cbe6d45be104dfaacc3afd241b940ca)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:57.619813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320615f2-d309-4730-8d17-244c0a8228ce admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:57.697290 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-320615f2-d309-4730-8d17-244c0a8228ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cbe6d45be104dfaacc3afd241b940ca', 'request_id': 'req-320615f2-d309-4730-8d17-244c0a8228ce'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:57.706397 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-320615f2-d309-4730-8d17-244c0a8228ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cbe6d45be104dfaacc3afd241b940ca', 'request_id': 'req-320615f2-d309-4730-8d17-244c0a8228ce'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:57.715585 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-320615f2-d309-4730-8d17-244c0a8228ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cbe6d45be104dfaacc3afd241b940ca', 'request_id': 'req-320615f2-d309-4730-8d17-244c0a8228ce'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:57.735358 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-320615f2-d309-4730-8d17-244c0a8228ce admin admin] The token cache is being invalidate because project 8cbe6d45be104dfaacc3afd241b940ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:05:57.735801 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-320615f2-d309-4730-8d17-244c0a8228ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:57.736567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-320615f2-d309-4730-8d17-244c0a8228ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:57.742716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 977/1930] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:05:57 2022] DELETE /identity/v3/projects/8cbe6d45be104dfaacc3afd241b940ca => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:57.744378 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04fd62b-ec07-4671-9b91-fdcab2905bf7 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:57.744670 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04fd62b-ec07-4671-9b91-fdcab2905bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:57.744951 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04fd62b-ec07-4671-9b91-fdcab2905bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:57.766619 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-d04fd62b-ec07-4671-9b91-fdcab2905bf7 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:57.768030 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-d04fd62b-ec07-4671-9b91-fdcab2905bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:57.788825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:57.860711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 954/1931] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:05:57.868478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30860e-35c7-4842-9074-fd968d767c5a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:57.868961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30860e-35c7-4842-9074-fd968d767c5a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:57.869054 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30860e-35c7-4842-9074-fd968d767c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:57.916612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ac30860e-35c7-4842-9074-fd968d767c5a None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:57.918652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ac30860e-35c7-4842-9074-fd968d767c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:57.931359 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 978/1932] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:57.937827 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04fd62b-ec07-4671-9b91-fdcab2905bf7 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:57.979821 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04fd62b-ec07-4671-9b91-fdcab2905bf7 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:58.014047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86eb11-752b-4302-93cb-0c0f7dcc5fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:58.015838 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86eb11-752b-4302-93cb-0c0f7dcc5fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:58.016796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86eb11-752b-4302-93cb-0c0f7dcc5fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:58.017273 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86eb11-752b-4302-93cb-0c0f7dcc5fd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:58.018759 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b86eb11-752b-4302-93cb-0c0f7dcc5fd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:58.019467 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b86eb11-752b-4302-93cb-0c0f7dcc5fd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:58.022609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b86eb11-752b-4302-93cb-0c0f7dcc5fd2 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:58.052158 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 955/1933] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:58.054594 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30860e-35c7-4842-9074-fd968d767c5a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:58.091552 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30860e-35c7-4842-9074-fd968d767c5a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:58.111868 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e84ef-92f8-4a19-8673-81c477ed27d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:58.112997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e84ef-92f8-4a19-8673-81c477ed27d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:58.113490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e84ef-92f8-4a19-8673-81c477ed27d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:58.113980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e84ef-92f8-4a19-8673-81c477ed27d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:58.115040 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882e84ef-92f8-4a19-8673-81c477ed27d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:58.117933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882e84ef-92f8-4a19-8673-81c477ed27d2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:58.135509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86eb11-752b-4302-93cb-0c0f7dcc5fd2 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:58.137092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-0b86eb11-752b-4302-93cb-0c0f7dcc5fd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:58.182253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86eb11-752b-4302-93cb-0c0f7dcc5fd2 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:58.194124 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 979/1934] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:05:58 2022] POST /identity/v3/projects => generated 361 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:58.200306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e84ef-92f8-4a19-8673-81c477ed27d2 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:58.206210 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af642ae3-ed09-457c-96b4-19393fabcd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:58.208263 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af642ae3-ed09-457c-96b4-19393fabcd80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:58.208762 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af642ae3-ed09-457c-96b4-19393fabcd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:58.209367 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af642ae3-ed09-457c-96b4-19393fabcd80 admin admin] PATH_INFO: `/v3/users/81b8f193d3354e9784dc62bbae040e72` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:58.214560 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af642ae3-ed09-457c-96b4-19393fabcd80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81b8f193d3354e9784dc62bbae040e72)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:58.219611 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:58.224070 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af642ae3-ed09-457c-96b4-19393fabcd80 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:58.303778 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-af642ae3-ed09-457c-96b4-19393fabcd80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81b8f193d3354e9784dc62bbae040e72'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:58.308183 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e84ef-92f8-4a19-8673-81c477ed27d2 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:58.312696 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-af642ae3-ed09-457c-96b4-19393fabcd80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81b8f193d3354e9784dc62bbae040e72'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:58.313720 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-af642ae3-ed09-457c-96b4-19393fabcd80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81b8f193d3354e9784dc62bbae040e72'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:58.314429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-af642ae3-ed09-457c-96b4-19393fabcd80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81b8f193d3354e9784dc62bbae040e72'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:58.315038 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ba1b9-be20-4473-82a9-90cd7321e377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:58.315832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ba1b9-be20-4473-82a9-90cd7321e377 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:58.316201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ba1b9-be20-4473-82a9-90cd7321e377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:58.316559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ba1b9-be20-4473-82a9-90cd7321e377 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:58.317384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ba1b9-be20-4473-82a9-90cd7321e377 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:58.319767 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ba1b9-be20-4473-82a9-90cd7321e377 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:58.323773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-af642ae3-ed09-457c-96b4-19393fabcd80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81b8f193d3354e9784dc62bbae040e72'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:58.343686 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 980/1935] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:05:58 2022] POST /identity/v3/users => generated 281 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:58.348482 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ba1b9-be20-4473-82a9-90cd7321e377 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:58.349978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d40ba1b9-be20-4473-82a9-90cd7321e377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:58.355318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 956/1936] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:05:58 2022] DELETE /identity/v3/users/81b8f193d3354e9784dc62bbae040e72 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:05:58.363746 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ba1b9-be20-4473-82a9-90cd7321e377 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:58.436078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bf697-1e81-4d7e-a52f-1e4c3422f129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:58.437301 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bf697-1e81-4d7e-a52f-1e4c3422f129 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:58.437602 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bf697-1e81-4d7e-a52f-1e4c3422f129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:58.437889 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bf697-1e81-4d7e-a52f-1e4c3422f129 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:58.438814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910bf697-1e81-4d7e-a52f-1e4c3422f129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:58.441155 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910bf697-1e81-4d7e-a52f-1e4c3422f129 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:58.463756 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 981/1937] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:58.467866 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af642ae3-ed09-457c-96b4-19393fabcd80 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:58.478365 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af642ae3-ed09-457c-96b4-19393fabcd80 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:58.483758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9497f0b-1e57-4d29-b7fa-a03d35e65bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:58.484559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9497f0b-1e57-4d29-b7fa-a03d35e65bf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:58.484846 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9497f0b-1e57-4d29-b7fa-a03d35e65bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:58.485124 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9497f0b-1e57-4d29-b7fa-a03d35e65bf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:58.485866 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9497f0b-1e57-4d29-b7fa-a03d35e65bf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:58.487579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9497f0b-1e57-4d29-b7fa-a03d35e65bf5 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:58.502283 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 957/1938] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:58.504387 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bf697-1e81-4d7e-a52f-1e4c3422f129 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:58.521774 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bf697-1e81-4d7e-a52f-1e4c3422f129 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:58.532560 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569771e0-2270-4ae3-a973-26a32d15b8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:58.535360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569771e0-2270-4ae3-a973-26a32d15b8a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:58.536899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569771e0-2270-4ae3-a973-26a32d15b8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:58.537827 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569771e0-2270-4ae3-a973-26a32d15b8a5 admin admin] PATH_INFO: `/v3/projects/70f1e87647d94fd1bd4b0c6e4c91640d/users/f400a5d595c74957a753699b6cee3563/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:58.569825 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569771e0-2270-4ae3-a973-26a32d15b8a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70f1e87647d94fd1bd4b0c6e4c91640d, user_id=f400a5d595c74957a753699b6cee3563, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:58.572774 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569771e0-2270-4ae3-a973-26a32d15b8a5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:58.591370 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 982/1939] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:58 2022] PUT /identity/v3/projects/70f1e87647d94fd1bd4b0c6e4c91640d/users/f400a5d595c74957a753699b6cee3563/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:58.601856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb4679-662b-4be7-ad1d-911965d833e3 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:58.602151 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb4679-662b-4be7-ad1d-911965d833e3 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:58.602692 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb4679-662b-4be7-ad1d-911965d833e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:58.628310 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569771e0-2270-4ae3-a973-26a32d15b8a5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:58.631549 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-569771e0-2270-4ae3-a973-26a32d15b8a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:58.642823 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-c5cb4679-662b-4be7-ad1d-911965d833e3 None None] MFA Rules not processed for user `f400a5d595c74957a753699b6cee3563`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:58.644234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-c5cb4679-662b-4be7-ad1d-911965d833e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:58.732614 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569771e0-2270-4ae3-a973-26a32d15b8a5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:58.738108 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01adf92-f562-4521-9468-22ecaf25afc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:58.739037 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01adf92-f562-4521-9468-22ecaf25afc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:58.739329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01adf92-f562-4521-9468-22ecaf25afc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:58.739607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01adf92-f562-4521-9468-22ecaf25afc0 admin admin] PATH_INFO: `/v3/projects/6b746bd093c749c5a2ffe6fd3d3e0417` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:58.740899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01adf92-f562-4521-9468-22ecaf25afc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b746bd093c749c5a2ffe6fd3d3e0417)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:58.742953 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01adf92-f562-4521-9468-22ecaf25afc0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:58.752805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:58.759661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:58.763493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 958/1940] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:58 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:58.810667 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb4679-662b-4be7-ad1d-911965d833e3 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:58.815003 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-c5cb4679-662b-4be7-ad1d-911965d833e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:58.837620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d01adf92-f562-4521-9468-22ecaf25afc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b746bd093c749c5a2ffe6fd3d3e0417', 'request_id': 'req-d01adf92-f562-4521-9468-22ecaf25afc0'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:58.860398 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d01adf92-f562-4521-9468-22ecaf25afc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b746bd093c749c5a2ffe6fd3d3e0417', 'request_id': 'req-d01adf92-f562-4521-9468-22ecaf25afc0'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:58.877399 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d01adf92-f562-4521-9468-22ecaf25afc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b746bd093c749c5a2ffe6fd3d3e0417', 'request_id': 'req-d01adf92-f562-4521-9468-22ecaf25afc0'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:58.903393 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d01adf92-f562-4521-9468-22ecaf25afc0 admin admin] The token cache is being invalidate because project 6b746bd093c749c5a2ffe6fd3d3e0417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:05:58.903680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d01adf92-f562-4521-9468-22ecaf25afc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:58.904802 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d01adf92-f562-4521-9468-22ecaf25afc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:58.906745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 983/1941] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:05:58 2022] DELETE /identity/v3/projects/6b746bd093c749c5a2ffe6fd3d3e0417 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:58.912051 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d330f38-f0fd-4c49-9f1c-e40f6b280c91 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:58.912753 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d330f38-f0fd-4c49-9f1c-e40f6b280c91 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:58.913212 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d330f38-f0fd-4c49-9f1c-e40f6b280c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:58.928401 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb4679-662b-4be7-ad1d-911965d833e3 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:58.935787 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5d330f38-f0fd-4c49-9f1c-e40f6b280c91 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:58.937630 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5d330f38-f0fd-4c49-9f1c-e40f6b280c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:59.004944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412706e-62bb-49b4-9880-215d6b83017f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:59.016301 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412706e-62bb-49b4-9880-215d6b83017f service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:59.016827 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412706e-62bb-49b4-9880-215d6b83017f service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:59.018346 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412706e-62bb-49b4-9880-215d6b83017f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:59.021704 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-d412706e-62bb-49b4-9880-215d6b83017f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:59.036685 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 984/1942] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:59.044980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d2bc1-4c4a-4bdd-a3e0-f1003829b0f8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:59.045750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d2bc1-4c4a-4bdd-a3e0-f1003829b0f8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:59.045750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d2bc1-4c4a-4bdd-a3e0-f1003829b0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:59.070669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-522d2bc1-4c4a-4bdd-a3e0-f1003829b0f8 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:59.072865 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-522d2bc1-4c4a-4bdd-a3e0-f1003829b0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:59.091206 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 985/1943] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:05:59.091783 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d2bc1-4c4a-4bdd-a3e0-f1003829b0f8 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:59.093680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-522d2bc1-4c4a-4bdd-a3e0-f1003829b0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:59.112811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d2bc1-4c4a-4bdd-a3e0-f1003829b0f8 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:59.121521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c38a6-b00c-4424-96dd-bfdcb1c20d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:59.123020 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c38a6-b00c-4424-96dd-bfdcb1c20d77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:59.123500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c38a6-b00c-4424-96dd-bfdcb1c20d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:59.125310 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c38a6-b00c-4424-96dd-bfdcb1c20d77 admin admin] PATH_INFO: `/v3/users/173a39e8e22e478ababe1341fb119b73` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:59.132489 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c38a6-b00c-4424-96dd-bfdcb1c20d77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=173a39e8e22e478ababe1341fb119b73)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:59.132651 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412706e-62bb-49b4-9880-215d6b83017f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:59.133256 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c38a6-b00c-4424-96dd-bfdcb1c20d77 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:59.133912 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412706e-62bb-49b4-9880-215d6b83017f service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:59.153886 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 959/1944] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:59.155733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412706e-62bb-49b4-9880-215d6b83017f service cinder] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:59.182988 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412706e-62bb-49b4-9880-215d6b83017f service cinder] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:59.190904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eecb0e8-81a8-4bd1-9236-2fc79c80758e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:59.192846 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecb0e8-81a8-4bd1-9236-2fc79c80758e admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:59.193238 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecb0e8-81a8-4bd1-9236-2fc79c80758e admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:59.193715 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecb0e8-81a8-4bd1-9236-2fc79c80758e admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:59.194606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eecb0e8-81a8-4bd1-9236-2fc79c80758e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:05:59.194965 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eecb0e8-81a8-4bd1-9236-2fc79c80758e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:59.196623 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eecb0e8-81a8-4bd1-9236-2fc79c80758e admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:59.211382 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 960/1945] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:05:59.214960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eecb0e8-81a8-4bd1-9236-2fc79c80758e admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:59.214960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-9eecb0e8-81a8-4bd1-9236-2fc79c80758e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:59.222161 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-190c38a6-b00c-4424-96dd-bfdcb1c20d77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '173a39e8e22e478ababe1341fb119b73'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:59.229483 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eecb0e8-81a8-4bd1-9236-2fc79c80758e admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:59.231881 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-190c38a6-b00c-4424-96dd-bfdcb1c20d77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '173a39e8e22e478ababe1341fb119b73'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:59.236349 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f5d57-b084-4209-8122-b78d06f80184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:59.237258 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f5d57-b084-4209-8122-b78d06f80184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:59.237549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f5d57-b084-4209-8122-b78d06f80184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:59.237890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f5d57-b084-4209-8122-b78d06f80184 admin admin] PATH_INFO: `/v3/users/f400a5d595c74957a753699b6cee3563` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:59.239172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f5d57-b084-4209-8122-b78d06f80184 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f400a5d595c74957a753699b6cee3563)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:59.241226 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f5d57-b084-4209-8122-b78d06f80184 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:59.249275 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-190c38a6-b00c-4424-96dd-bfdcb1c20d77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '173a39e8e22e478ababe1341fb119b73'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:59.249489 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-190c38a6-b00c-4424-96dd-bfdcb1c20d77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '173a39e8e22e478ababe1341fb119b73'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:59.250633 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-190c38a6-b00c-4424-96dd-bfdcb1c20d77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '173a39e8e22e478ababe1341fb119b73'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:59.273498 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 986/1946] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:05:59 2022] DELETE /identity/v3/users/173a39e8e22e478ababe1341fb119b73 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:59.273498 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c38a6-b00c-4424-96dd-bfdcb1c20d77 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:59.273498 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-190c38a6-b00c-4424-96dd-bfdcb1c20d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:59.286586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:59.327093 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-a10f5d57-b084-4209-8122-b78d06f80184 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f400a5d595c74957a753699b6cee3563'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:59.335933 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:59.345071 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-a10f5d57-b084-4209-8122-b78d06f80184 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f400a5d595c74957a753699b6cee3563'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:59.347447 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-a10f5d57-b084-4209-8122-b78d06f80184 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f400a5d595c74957a753699b6cee3563'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:59.349107 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-a10f5d57-b084-4209-8122-b78d06f80184 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f400a5d595c74957a753699b6cee3563'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:59.363550 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-a10f5d57-b084-4209-8122-b78d06f80184 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f400a5d595c74957a753699b6cee3563'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:59.389056 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c38a6-b00c-4424-96dd-bfdcb1c20d77 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:59.401151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03395eb5-13f3-4793-ad99-a3fb90c0197a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:59.402111 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03395eb5-13f3-4793-ad99-a3fb90c0197a admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:59.403805 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03395eb5-13f3-4793-ad99-a3fb90c0197a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:59.404103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03395eb5-13f3-4793-ad99-a3fb90c0197a admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:59.405352 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03395eb5-13f3-4793-ad99-a3fb90c0197a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:59.408057 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03395eb5-13f3-4793-ad99-a3fb90c0197a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:59.411350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 961/1947] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:05:59 2022] DELETE /identity/v3/users/f400a5d595c74957a753699b6cee3563 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:59.415866 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e1543-476c-4791-bf43-ae1688480a7e None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:59.416191 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e1543-476c-4791-bf43-ae1688480a7e None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:59.416448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e1543-476c-4791-bf43-ae1688480a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:59.442734 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-9c1e1543-476c-4791-bf43-ae1688480a7e None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:59.444224 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-9c1e1543-476c-4791-bf43-ae1688480a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:59.447251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 987/1948] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:05:59 2022] POST /identity/v3/projects => generated 363 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:59.449368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3055c1-988d-44d7-b0c0-235dd3c79c0e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:59.449695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3055c1-988d-44d7-b0c0-235dd3c79c0e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:59.450253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3055c1-988d-44d7-b0c0-235dd3c79c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:59.476309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2f3055c1-988d-44d7-b0c0-235dd3c79c0e None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:05:59.478173 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2f3055c1-988d-44d7-b0c0-235dd3c79c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:59.578873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 988/1949] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:59.578873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3055c1-988d-44d7-b0c0-235dd3c79c0e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:59.580584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2f3055c1-988d-44d7-b0c0-235dd3c79c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:59.587690 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 962/1950] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:05:59.594087 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3055c1-988d-44d7-b0c0-235dd3c79c0e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:59.600509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9cc7c6-4878-4f3d-92b6-2bb6bbc41512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:59.601278 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cc7c6-4878-4f3d-92b6-2bb6bbc41512 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:59.601625 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cc7c6-4878-4f3d-92b6-2bb6bbc41512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:59.601971 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e1543-476c-4791-bf43-ae1688480a7e None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:59.602140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cc7c6-4878-4f3d-92b6-2bb6bbc41512 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:59.602980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9cc7c6-4878-4f3d-92b6-2bb6bbc41512 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:59.603637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-9c1e1543-476c-4791-bf43-ae1688480a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:59.605104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9cc7c6-4878-4f3d-92b6-2bb6bbc41512 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:59.650796 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 989/1951] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:05:59 2022] POST /identity/v3/users => generated 282 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:05:59.650796 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9cc7c6-4878-4f3d-92b6-2bb6bbc41512 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:59.653639 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6d9cc7c6-4878-4f3d-92b6-2bb6bbc41512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:59.720608 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e1543-476c-4791-bf43-ae1688480a7e None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:59.727064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3acc2c-7efc-4928-adbd-f1ed76fae2a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:59.727742 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3acc2c-7efc-4928-adbd-f1ed76fae2a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:59.728034 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3acc2c-7efc-4928-adbd-f1ed76fae2a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:59.728316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3acc2c-7efc-4928-adbd-f1ed76fae2a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:59.746532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3acc2c-7efc-4928-adbd-f1ed76fae2a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:59.748278 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3acc2c-7efc-4928-adbd-f1ed76fae2a4 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:59.752396 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9cc7c6-4878-4f3d-92b6-2bb6bbc41512 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:59.759617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7887b6-64e0-4fa7-9d83-7b24d679cc02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:59.759861 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 963/1952] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:59.760273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7887b6-64e0-4fa7-9d83-7b24d679cc02 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:59.760561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7887b6-64e0-4fa7-9d83-7b24d679cc02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:59.760838 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7887b6-64e0-4fa7-9d83-7b24d679cc02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:59.761218 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3acc2c-7efc-4928-adbd-f1ed76fae2a4 service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:59.771669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7887b6-64e0-4fa7-9d83-7b24d679cc02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:59.772452 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3acc2c-7efc-4928-adbd-f1ed76fae2a4 service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:59.773392 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7887b6-64e0-4fa7-9d83-7b24d679cc02 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:59.783432 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc617d-08c0-4aa5-ad5d-2f4e6378c013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:59.784350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc617d-08c0-4aa5-ad5d-2f4e6378c013 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:59.784644 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc617d-08c0-4aa5-ad5d-2f4e6378c013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:59.784923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc617d-08c0-4aa5-ad5d-2f4e6378c013 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:59.785654 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc617d-08c0-4aa5-ad5d-2f4e6378c013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:59.786409 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 990/1953] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:05:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:05:59.787225 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7887b6-64e0-4fa7-9d83-7b24d679cc02 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:59.788037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc617d-08c0-4aa5-ad5d-2f4e6378c013 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:59.788853 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-af7887b6-64e0-4fa7-9d83-7b24d679cc02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:05:59.802318 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7887b6-64e0-4fa7-9d83-7b24d679cc02 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:59.805718 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 964/1954] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:59.808269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583f0e2-10c0-4a80-9986-c333d8ae5370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:59.809389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583f0e2-10c0-4a80-9986-c333d8ae5370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:59.809656 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583f0e2-10c0-4a80-9986-c333d8ae5370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:59.809769 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc617d-08c0-4aa5-ad5d-2f4e6378c013 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:59.809932 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583f0e2-10c0-4a80-9986-c333d8ae5370 admin admin] PATH_INFO: `/v3/users/fa4c392318044e78bf8ccd03ce203bef` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:59.811263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583f0e2-10c0-4a80-9986-c333d8ae5370 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa4c392318044e78bf8ccd03ce203bef)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:59.814139 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583f0e2-10c0-4a80-9986-c333d8ae5370 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:59.831421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc617d-08c0-4aa5-ad5d-2f4e6378c013 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:59.836101 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:05:59.840134 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f442f3f-4792-4f47-b559-1389c56e5f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:59.841061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f442f3f-4792-4f47-b559-1389c56e5f79 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:59.841390 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f442f3f-4792-4f47-b559-1389c56e5f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:59.841699 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f442f3f-4792-4f47-b559-1389c56e5f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:59.844208 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f442f3f-4792-4f47-b559-1389c56e5f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:59.846818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f442f3f-4792-4f47-b559-1389c56e5f79 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:59.865575 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 965/1955] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:05:59.867763 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f442f3f-4792-4f47-b559-1389c56e5f79 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:05:59.899642 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f442f3f-4792-4f47-b559-1389c56e5f79 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:05:59.903108 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5583f0e2-10c0-4a80-9986-c333d8ae5370 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa4c392318044e78bf8ccd03ce203bef'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:59.912507 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f543449-d8aa-4c24-a2b9-a6b5fbc64200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:05:59.914880 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f543449-d8aa-4c24-a2b9-a6b5fbc64200 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:59.915214 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5583f0e2-10c0-4a80-9986-c333d8ae5370 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa4c392318044e78bf8ccd03ce203bef'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:59.915593 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f543449-d8aa-4c24-a2b9-a6b5fbc64200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:59.916106 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f543449-d8aa-4c24-a2b9-a6b5fbc64200 admin admin] PATH_INFO: `/v3/projects/92bbac43c7b34d73be60ddcde92ae3fd/users/35c3f499f55444a994cfe41460a6b1e1/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:05:59.935262 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5583f0e2-10c0-4a80-9986-c333d8ae5370 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa4c392318044e78bf8ccd03ce203bef'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:59.937639 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5583f0e2-10c0-4a80-9986-c333d8ae5370 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa4c392318044e78bf8ccd03ce203bef'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:59.938711 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5583f0e2-10c0-4a80-9986-c333d8ae5370 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa4c392318044e78bf8ccd03ce203bef'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:05:59.944036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f543449-d8aa-4c24-a2b9-a6b5fbc64200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92bbac43c7b34d73be60ddcde92ae3fd, user_id=35c3f499f55444a994cfe41460a6b1e1, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:05:59.947776 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f543449-d8aa-4c24-a2b9-a6b5fbc64200 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:05:59.955283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 991/1956] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:05:59 2022] DELETE /identity/v3/users/fa4c392318044e78bf8ccd03ce203bef => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:59.971194 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 966/1957] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:05:59 2022] PUT /identity/v3/projects/92bbac43c7b34d73be60ddcde92ae3fd/users/35c3f499f55444a994cfe41460a6b1e1/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:05:59.982039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee67443-0a39-41da-987e-6e35e27f4655 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:05:59.982039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee67443-0a39-41da-987e-6e35e27f4655 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:05:59.982361 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee67443-0a39-41da-987e-6e35e27f4655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:00.018850 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3ee67443-0a39-41da-987e-6e35e27f4655 None None] MFA Rules not processed for user `35c3f499f55444a994cfe41460a6b1e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:00.021314 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3ee67443-0a39-41da-987e-6e35e27f4655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:00.190967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 992/1958] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:05:59 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:00.197321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f543449-d8aa-4c24-a2b9-a6b5fbc64200 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:00.200532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-0f543449-d8aa-4c24-a2b9-a6b5fbc64200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:00.276953 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:00.277469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee67443-0a39-41da-987e-6e35e27f4655 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:00.281040 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3ee67443-0a39-41da-987e-6e35e27f4655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:00.397287 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f543449-d8aa-4c24-a2b9-a6b5fbc64200 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:00.412885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7552c-3860-4f0b-9b3a-38638a0e7f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:00.413571 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7552c-3860-4f0b-9b3a-38638a0e7f4c admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:00.413863 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7552c-3860-4f0b-9b3a-38638a0e7f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:00.414141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7552c-3860-4f0b-9b3a-38638a0e7f4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:00.414886 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7552c-3860-4f0b-9b3a-38638a0e7f4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:00.416843 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7552c-3860-4f0b-9b3a-38638a0e7f4c admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:00.428072 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee67443-0a39-41da-987e-6e35e27f4655 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:00.432775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6052a-32e5-4319-b9b1-11595ea875d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:00.433799 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6052a-32e5-4319-b9b1-11595ea875d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:00.434009 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6052a-32e5-4319-b9b1-11595ea875d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:00.434287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6052a-32e5-4319-b9b1-11595ea875d9 admin admin] PATH_INFO: `/v3/projects/70f1e87647d94fd1bd4b0c6e4c91640d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:00.435634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6052a-32e5-4319-b9b1-11595ea875d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70f1e87647d94fd1bd4b0c6e4c91640d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:00.438326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6052a-32e5-4319-b9b1-11595ea875d9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:00.445809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 967/1959] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:00 2022] POST /identity/v3/projects => generated 365 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:00.446688 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7552c-3860-4f0b-9b3a-38638a0e7f4c admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:00.448319 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-5fa7552c-3860-4f0b-9b3a-38638a0e7f4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:00.469949 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:00.505265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bbc6052a-32e5-4319-b9b1-11595ea875d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70f1e87647d94fd1bd4b0c6e4c91640d', 'request_id': 'req-bbc6052a-32e5-4319-b9b1-11595ea875d9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:00.512392 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bbc6052a-32e5-4319-b9b1-11595ea875d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70f1e87647d94fd1bd4b0c6e4c91640d', 'request_id': 'req-bbc6052a-32e5-4319-b9b1-11595ea875d9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:00.521304 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bbc6052a-32e5-4319-b9b1-11595ea875d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70f1e87647d94fd1bd4b0c6e4c91640d', 'request_id': 'req-bbc6052a-32e5-4319-b9b1-11595ea875d9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:00.521547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7552c-3860-4f0b-9b3a-38638a0e7f4c admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:00.542516 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bbc6052a-32e5-4319-b9b1-11595ea875d9 admin admin] The token cache is being invalidate because project 70f1e87647d94fd1bd4b0c6e4c91640d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:00.542830 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bbc6052a-32e5-4319-b9b1-11595ea875d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:00.543629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bbc6052a-32e5-4319-b9b1-11595ea875d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:00.545526 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 993/1960] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:00 2022] DELETE /identity/v3/projects/70f1e87647d94fd1bd4b0c6e4c91640d => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:00.547294 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6052a-32e5-4319-b9b1-11595ea875d9 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:00.548802 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bbc6052a-32e5-4319-b9b1-11595ea875d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:00.594221 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161ad5a-e9b1-44e4-b79f-6cbdb1c5376f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:00.595313 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161ad5a-e9b1-44e4-b79f-6cbdb1c5376f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:00.595601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161ad5a-e9b1-44e4-b79f-6cbdb1c5376f admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:00.595879 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161ad5a-e9b1-44e4-b79f-6cbdb1c5376f admin admin] PATH_INFO: `/v3/projects/563df53d3d894b6db3c703a076b9710a` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:00.597198 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161ad5a-e9b1-44e4-b79f-6cbdb1c5376f admin admin] RBAC: Authorizing `identity:delete_project(project_id=563df53d3d894b6db3c703a076b9710a)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:00.599381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161ad5a-e9b1-44e4-b79f-6cbdb1c5376f admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:00.651948 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6052a-32e5-4319-b9b1-11595ea875d9 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:00.663209 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f161ad5a-e9b1-44e4-b79f-6cbdb1c5376f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '563df53d3d894b6db3c703a076b9710a', 'request_id': 'req-f161ad5a-e9b1-44e4-b79f-6cbdb1c5376f'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:00.672059 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f161ad5a-e9b1-44e4-b79f-6cbdb1c5376f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '563df53d3d894b6db3c703a076b9710a', 'request_id': 'req-f161ad5a-e9b1-44e4-b79f-6cbdb1c5376f'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:00.677729 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:00.680186 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f161ad5a-e9b1-44e4-b79f-6cbdb1c5376f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '563df53d3d894b6db3c703a076b9710a', 'request_id': 'req-f161ad5a-e9b1-44e4-b79f-6cbdb1c5376f'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:00.706289 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f161ad5a-e9b1-44e4-b79f-6cbdb1c5376f admin admin] The token cache is being invalidate because project 563df53d3d894b6db3c703a076b9710a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:00.706778 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f161ad5a-e9b1-44e4-b79f-6cbdb1c5376f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:00.708024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f161ad5a-e9b1-44e4-b79f-6cbdb1c5376f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:00.711884 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 968/1961] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:00 2022] DELETE /identity/v3/projects/563df53d3d894b6db3c703a076b9710a => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:00.712865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161ad5a-e9b1-44e4-b79f-6cbdb1c5376f admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:00.715790 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f161ad5a-e9b1-44e4-b79f-6cbdb1c5376f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:00.740679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592682d7-5bf5-4c9f-8130-895d6f5da8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:00.741573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592682d7-5bf5-4c9f-8130-895d6f5da8fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:00.741859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592682d7-5bf5-4c9f-8130-895d6f5da8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:00.742137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592682d7-5bf5-4c9f-8130-895d6f5da8fb admin admin] PATH_INFO: `/v3/projects/369bcd27406d43ed930e94e255ff8dc1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:00.743474 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592682d7-5bf5-4c9f-8130-895d6f5da8fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=369bcd27406d43ed930e94e255ff8dc1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:00.745457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592682d7-5bf5-4c9f-8130-895d6f5da8fb admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:00.811067 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161ad5a-e9b1-44e4-b79f-6cbdb1c5376f admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:00.815222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-592682d7-5bf5-4c9f-8130-895d6f5da8fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '369bcd27406d43ed930e94e255ff8dc1', 'request_id': 'req-592682d7-5bf5-4c9f-8130-895d6f5da8fb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:00.822571 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-592682d7-5bf5-4c9f-8130-895d6f5da8fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '369bcd27406d43ed930e94e255ff8dc1', 'request_id': 'req-592682d7-5bf5-4c9f-8130-895d6f5da8fb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:00.824113 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:00.832301 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-592682d7-5bf5-4c9f-8130-895d6f5da8fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '369bcd27406d43ed930e94e255ff8dc1', 'request_id': 'req-592682d7-5bf5-4c9f-8130-895d6f5da8fb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:00.850855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-592682d7-5bf5-4c9f-8130-895d6f5da8fb admin admin] The token cache is being invalidate because project 369bcd27406d43ed930e94e255ff8dc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:00.851133 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-592682d7-5bf5-4c9f-8130-895d6f5da8fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:00.851838 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-592682d7-5bf5-4c9f-8130-895d6f5da8fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:00.853502 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 994/1962] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:00 2022] DELETE /identity/v3/projects/369bcd27406d43ed930e94e255ff8dc1 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:00.858739 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cef659-1124-4742-ab1d-dcd8fc9f0949 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:00.858739 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cef659-1124-4742-ab1d-dcd8fc9f0949 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:00.858739 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cef659-1124-4742-ab1d-dcd8fc9f0949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:00.884358 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-45cef659-1124-4742-ab1d-dcd8fc9f0949 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:00.886077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-45cef659-1124-4742-ab1d-dcd8fc9f0949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:00.889584 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d465ff6-d1db-40ea-88c8-4544e28dc29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:00.891080 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d465ff6-d1db-40ea-88c8-4544e28dc29c admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:00.891578 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d465ff6-d1db-40ea-88c8-4544e28dc29c admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:00.892079 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d465ff6-d1db-40ea-88c8-4544e28dc29c admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:00.893418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d465ff6-d1db-40ea-88c8-4544e28dc29c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:00.897362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d465ff6-d1db-40ea-88c8-4544e28dc29c admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:00.898001 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 995/1963] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:00.898945 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cef659-1124-4742-ab1d-dcd8fc9f0949 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:00.906952 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-45cef659-1124-4742-ab1d-dcd8fc9f0949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:00.922114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cef659-1124-4742-ab1d-dcd8fc9f0949 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:00.927772 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5251f4-ef5a-4a21-a369-a2bd6bc06bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:00.928682 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5251f4-ef5a-4a21-a369-a2bd6bc06bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:00.928967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5251f4-ef5a-4a21-a369-a2bd6bc06bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:00.929242 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5251f4-ef5a-4a21-a369-a2bd6bc06bd2 admin admin] PATH_INFO: `/v3/users/c3f20ea085e34f748a8aee94300351cd` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:00.930547 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5251f4-ef5a-4a21-a369-a2bd6bc06bd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3f20ea085e34f748a8aee94300351cd)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:00.930675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 969/1964] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:00 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:00.932396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d465ff6-d1db-40ea-88c8-4544e28dc29c admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:00.932524 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5251f4-ef5a-4a21-a369-a2bd6bc06bd2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:00.935645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-9d465ff6-d1db-40ea-88c8-4544e28dc29c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:00.957536 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d465ff6-d1db-40ea-88c8-4544e28dc29c admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:00.966560 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd4f5a-cda5-49ac-8246-bbda9ee7a131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:00.968964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd4f5a-cda5-49ac-8246-bbda9ee7a131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:00.969272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd4f5a-cda5-49ac-8246-bbda9ee7a131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:00.969792 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd4f5a-cda5-49ac-8246-bbda9ee7a131 admin admin] PATH_INFO: `/v3/users/b9cb4a9a3de342ad8941e1b2a00c03dd` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:00.972266 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd4f5a-cda5-49ac-8246-bbda9ee7a131 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9cb4a9a3de342ad8941e1b2a00c03dd)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:00.976427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd4f5a-cda5-49ac-8246-bbda9ee7a131 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:01.016903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ec5251f4-ef5a-4a21-a369-a2bd6bc06bd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3f20ea085e34f748a8aee94300351cd'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:01.028105 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ec5251f4-ef5a-4a21-a369-a2bd6bc06bd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3f20ea085e34f748a8aee94300351cd'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:01.044138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ec5251f4-ef5a-4a21-a369-a2bd6bc06bd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3f20ea085e34f748a8aee94300351cd'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:01.044893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ec5251f4-ef5a-4a21-a369-a2bd6bc06bd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3f20ea085e34f748a8aee94300351cd'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:01.045544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ec5251f4-ef5a-4a21-a369-a2bd6bc06bd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3f20ea085e34f748a8aee94300351cd'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:01.048667 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-00cd4f5a-cda5-49ac-8246-bbda9ee7a131 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9cb4a9a3de342ad8941e1b2a00c03dd'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:01.057223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-00cd4f5a-cda5-49ac-8246-bbda9ee7a131 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9cb4a9a3de342ad8941e1b2a00c03dd'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:01.058363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-00cd4f5a-cda5-49ac-8246-bbda9ee7a131 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9cb4a9a3de342ad8941e1b2a00c03dd'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:01.059498 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-00cd4f5a-cda5-49ac-8246-bbda9ee7a131 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9cb4a9a3de342ad8941e1b2a00c03dd'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:01.063124 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 996/1965] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:00 2022] DELETE /identity/v3/users/c3f20ea085e34f748a8aee94300351cd => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:01.064331 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc8058-5e33-42b7-868e-f6cb4cb1556f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:01.064636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc8058-5e33-42b7-868e-f6cb4cb1556f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:01.064929 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc8058-5e33-42b7-868e-f6cb4cb1556f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:01.073989 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-00cd4f5a-cda5-49ac-8246-bbda9ee7a131 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9cb4a9a3de342ad8941e1b2a00c03dd'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:01.087703 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-dcbc8058-5e33-42b7-868e-f6cb4cb1556f None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:01.089089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-dcbc8058-5e33-42b7-868e-f6cb4cb1556f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:01.100125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:01.110891 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 970/1966] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:00 2022] DELETE /identity/v3/users/b9cb4a9a3de342ad8941e1b2a00c03dd => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:01.116928 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd4f5a-cda5-49ac-8246-bbda9ee7a131 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:01.119981 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-00cd4f5a-cda5-49ac-8246-bbda9ee7a131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:01.206476 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 997/1967] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:01.207959 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc8058-5e33-42b7-868e-f6cb4cb1556f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:01.223860 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc8058-5e33-42b7-868e-f6cb4cb1556f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:01.263197 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd4f5a-cda5-49ac-8246-bbda9ee7a131 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:01.274780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784ed53-3c9a-448d-b2b3-a28b357ce3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:01.276058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784ed53-3c9a-448d-b2b3-a28b357ce3ac admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:01.276524 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784ed53-3c9a-448d-b2b3-a28b357ce3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:01.276687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784ed53-3c9a-448d-b2b3-a28b357ce3ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:01.277574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784ed53-3c9a-448d-b2b3-a28b357ce3ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:01.281276 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784ed53-3c9a-448d-b2b3-a28b357ce3ac admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:01.317167 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 971/1968] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:01.324060 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784ed53-3c9a-448d-b2b3-a28b357ce3ac admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:01.334408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:01.350534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784ed53-3c9a-448d-b2b3-a28b357ce3ac admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:01.354113 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fb073-e4d1-47b9-aad2-288ebaa9ba58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:01.355562 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fb073-e4d1-47b9-aad2-288ebaa9ba58 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:01.356266 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fb073-e4d1-47b9-aad2-288ebaa9ba58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:01.356718 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fb073-e4d1-47b9-aad2-288ebaa9ba58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:01.360762 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2fb073-e4d1-47b9-aad2-288ebaa9ba58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:06:01.361137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2fb073-e4d1-47b9-aad2-288ebaa9ba58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:01.366516 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a7c95-8308-4893-a1e1-147c6aa10703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:01.368140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a7c95-8308-4893-a1e1-147c6aa10703 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:01.369256 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a7c95-8308-4893-a1e1-147c6aa10703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:01.369256 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a7c95-8308-4893-a1e1-147c6aa10703 admin admin] PATH_INFO: `/v3/projects/a0b4de8454c44e448b78c0ad666f4b76/users/23abea28d5aa4265b355f4980354498f/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:01.372075 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2fb073-e4d1-47b9-aad2-288ebaa9ba58 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:01.397865 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 998/1969] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:06:01.399305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fb073-e4d1-47b9-aad2-288ebaa9ba58 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:01.404810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3b2fb073-e4d1-47b9-aad2-288ebaa9ba58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:01.406318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a7c95-8308-4893-a1e1-147c6aa10703 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0b4de8454c44e448b78c0ad666f4b76, user_id=23abea28d5aa4265b355f4980354498f, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:01.410718 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a7c95-8308-4893-a1e1-147c6aa10703 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:01.434645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 972/1970] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:01 2022] PUT /identity/v3/projects/a0b4de8454c44e448b78c0ad666f4b76/users/23abea28d5aa4265b355f4980354498f/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:01.435853 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a7c95-8308-4893-a1e1-147c6aa10703 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:01.445317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fb073-e4d1-47b9-aad2-288ebaa9ba58 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:01.548505 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0fee5-53dc-41a0-bcff-43a336bfd0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:01.549516 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0fee5-53dc-41a0-bcff-43a336bfd0ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:01.550100 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0fee5-53dc-41a0-bcff-43a336bfd0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:01.550607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0fee5-53dc-41a0-bcff-43a336bfd0ce admin admin] PATH_INFO: `/v3/projects/21903e884eed4f2cb0618ac7b2a2c9d4` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:01.552790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a0fee5-53dc-41a0-bcff-43a336bfd0ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=21903e884eed4f2cb0618ac7b2a2c9d4)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:01.553258 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a7c95-8308-4893-a1e1-147c6aa10703 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:01.555332 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a0fee5-53dc-41a0-bcff-43a336bfd0ce admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:01.560581 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3017a-e966-4266-87c3-a9faa4feeade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:01.561286 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3017a-e966-4266-87c3-a9faa4feeade admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:01.561569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3017a-e966-4266-87c3-a9faa4feeade admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:01.562419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3017a-e966-4266-87c3-a9faa4feeade admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:01.563854 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3017a-e966-4266-87c3-a9faa4feeade admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:01.565138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3017a-e966-4266-87c3-a9faa4feeade admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:01.612807 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 973/1971] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:01 2022] POST /identity/v3/projects => generated 385 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:01.613846 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3017a-e966-4266-87c3-a9faa4feeade admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:01.650655 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-89a0fee5-53dc-41a0-bcff-43a336bfd0ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21903e884eed4f2cb0618ac7b2a2c9d4', 'request_id': 'req-89a0fee5-53dc-41a0-bcff-43a336bfd0ce'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:01.658131 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-89a0fee5-53dc-41a0-bcff-43a336bfd0ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21903e884eed4f2cb0618ac7b2a2c9d4', 'request_id': 'req-89a0fee5-53dc-41a0-bcff-43a336bfd0ce'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:01.666255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-89a0fee5-53dc-41a0-bcff-43a336bfd0ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21903e884eed4f2cb0618ac7b2a2c9d4', 'request_id': 'req-89a0fee5-53dc-41a0-bcff-43a336bfd0ce'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:01.682274 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-89a0fee5-53dc-41a0-bcff-43a336bfd0ce admin admin] The token cache is being invalidate because project 21903e884eed4f2cb0618ac7b2a2c9d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:01.682599 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-89a0fee5-53dc-41a0-bcff-43a336bfd0ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:01.683329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-89a0fee5-53dc-41a0-bcff-43a336bfd0ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:01.684955 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 999/1972] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:01 2022] DELETE /identity/v3/projects/21903e884eed4f2cb0618ac7b2a2c9d4 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:01.690307 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0fee5-53dc-41a0-bcff-43a336bfd0ce admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:01.691767 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-89a0fee5-53dc-41a0-bcff-43a336bfd0ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:01.704728 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:01.711178 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3017a-e966-4266-87c3-a9faa4feeade admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:01.790883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0fee5-53dc-41a0-bcff-43a336bfd0ce admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:01.795794 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a70cb-797f-44fb-add2-9222dccd223a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:01.797262 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a70cb-797f-44fb-add2-9222dccd223a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:01.797505 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb703499-9209-4002-91c2-34f83d79f261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:01.797681 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a70cb-797f-44fb-add2-9222dccd223a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:01.797852 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a70cb-797f-44fb-add2-9222dccd223a admin admin] PATH_INFO: `/v3/projects/20aa79df21104b3994c5f658401c9446` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:01.798735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb703499-9209-4002-91c2-34f83d79f261 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:01.799167 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a70cb-797f-44fb-add2-9222dccd223a admin admin] RBAC: Authorizing `identity:delete_project(project_id=20aa79df21104b3994c5f658401c9446)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:01.799276 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb703499-9209-4002-91c2-34f83d79f261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:01.799728 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb703499-9209-4002-91c2-34f83d79f261 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:01.800639 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb703499-9209-4002-91c2-34f83d79f261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:01.801145 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a70cb-797f-44fb-add2-9222dccd223a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:01.803818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb703499-9209-4002-91c2-34f83d79f261 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:01.822643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 974/1973] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:01.826029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb703499-9209-4002-91c2-34f83d79f261 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:01.826162 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-fb703499-9209-4002-91c2-34f83d79f261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:01.828028 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:01.841390 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb703499-9209-4002-91c2-34f83d79f261 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:01.846738 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fee14b-9089-4b83-817f-282c663d3517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:01.847437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fee14b-9089-4b83-817f-282c663d3517 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:01.847724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fee14b-9089-4b83-817f-282c663d3517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:01.848006 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fee14b-9089-4b83-817f-282c663d3517 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:01.848712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fee14b-9089-4b83-817f-282c663d3517 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:01.850781 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fee14b-9089-4b83-817f-282c663d3517 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:01.860750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-495a70cb-797f-44fb-add2-9222dccd223a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20aa79df21104b3994c5f658401c9446', 'request_id': 'req-495a70cb-797f-44fb-add2-9222dccd223a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:01.868619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-495a70cb-797f-44fb-add2-9222dccd223a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20aa79df21104b3994c5f658401c9446', 'request_id': 'req-495a70cb-797f-44fb-add2-9222dccd223a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:01.873628 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 975/1974] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:01 2022] POST /identity/v3/users => generated 293 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:01.875117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6a4dd-62a9-460b-9670-55f7df15c90d None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:01.875406 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6a4dd-62a9-460b-9670-55f7df15c90d None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:01.875693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6a4dd-62a9-460b-9670-55f7df15c90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:01.876145 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-495a70cb-797f-44fb-add2-9222dccd223a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20aa79df21104b3994c5f658401c9446', 'request_id': 'req-495a70cb-797f-44fb-add2-9222dccd223a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:01.894183 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-495a70cb-797f-44fb-add2-9222dccd223a admin admin] The token cache is being invalidate because project 20aa79df21104b3994c5f658401c9446 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:01.894623 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-495a70cb-797f-44fb-add2-9222dccd223a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:01.895616 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-495a70cb-797f-44fb-add2-9222dccd223a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:01.897657 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1000/1975] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:01 2022] DELETE /identity/v3/projects/20aa79df21104b3994c5f658401c9446 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:01.904833 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a70cb-797f-44fb-add2-9222dccd223a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:01.905318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-cfa6a4dd-62a9-460b-9670-55f7df15c90d None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:01.905318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-cfa6a4dd-62a9-460b-9670-55f7df15c90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:01.907378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-495a70cb-797f-44fb-add2-9222dccd223a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:01.995856 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a70cb-797f-44fb-add2-9222dccd223a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:02.001914 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c31f80-41be-488d-ba0d-f95a6d444163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:02.002895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c31f80-41be-488d-ba0d-f95a6d444163 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:02.003138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c31f80-41be-488d-ba0d-f95a6d444163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:02.003374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 976/1976] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:02.003488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c31f80-41be-488d-ba0d-f95a6d444163 admin admin] PATH_INFO: `/v3/projects/a0b4de8454c44e448b78c0ad666f4b76/users/23abea28d5aa4265b355f4980354498f/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:02.004868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6a4dd-62a9-460b-9670-55f7df15c90d None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:02.006324 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c31f80-41be-488d-ba0d-f95a6d444163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0b4de8454c44e448b78c0ad666f4b76, user_id=23abea28d5aa4265b355f4980354498f, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:02.008040 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-cfa6a4dd-62a9-460b-9670-55f7df15c90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:02.010031 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c31f80-41be-488d-ba0d-f95a6d444163 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:02.024661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1001/1977] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:01 2022] PUT /identity/v3/projects/a0b4de8454c44e448b78c0ad666f4b76/users/23abea28d5aa4265b355f4980354498f/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:02.026260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c54d9-c57e-4a60-b447-0f721b73dad2 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:02.026633 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c54d9-c57e-4a60-b447-0f721b73dad2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:02.026911 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c54d9-c57e-4a60-b447-0f721b73dad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:02.036100 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6a4dd-62a9-460b-9670-55f7df15c90d None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:02.049490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4c1c54d9-c57e-4a60-b447-0f721b73dad2 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:02.050754 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4c1c54d9-c57e-4a60-b447-0f721b73dad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:02.119714 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bccb411-1802-4140-9eef-d88d123a1601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:02.120311 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccb411-1802-4140-9eef-d88d123a1601 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:02.120598 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccb411-1802-4140-9eef-d88d123a1601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:02.120876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccb411-1802-4140-9eef-d88d123a1601 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:02.121623 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bccb411-1802-4140-9eef-d88d123a1601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:02.123311 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bccb411-1802-4140-9eef-d88d123a1601 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:02.123726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1002/1978] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:02.125425 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996ddbf-6934-4d49-93ff-5a25868bad96 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:02.125713 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996ddbf-6934-4d49-93ff-5a25868bad96 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:02.125989 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996ddbf-6934-4d49-93ff-5a25868bad96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:02.146366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-7996ddbf-6934-4d49-93ff-5a25868bad96 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:02.147896 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7996ddbf-6934-4d49-93ff-5a25868bad96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:02.148036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 977/1979] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:02.149053 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c61e4c-7f4b-436f-952f-b23b82ebcb02 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:02.149358 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c61e4c-7f4b-436f-952f-b23b82ebcb02 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:02.149798 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c61e4c-7f4b-436f-952f-b23b82ebcb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:02.158303 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1003/1980] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:02.162829 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad9b92-4209-4944-b869-c59d276e880f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:02.163118 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad9b92-4209-4944-b869-c59d276e880f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:02.163398 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad9b92-4209-4944-b869-c59d276e880f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:02.181851 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-b6c61e4c-7f4b-436f-952f-b23b82ebcb02 None None] MFA Rules not processed for user `23abea28d5aa4265b355f4980354498f`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:02.183250 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-b6c61e4c-7f4b-436f-952f-b23b82ebcb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:02.189899 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:02.191244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a6ad9b92-4209-4944-b869-c59d276e880f None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:02.192696 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a6ad9b92-4209-4944-b869-c59d276e880f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:02.201908 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1004/1981] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:02.203387 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad9b92-4209-4944-b869-c59d276e880f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:02.214587 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad9b92-4209-4944-b869-c59d276e880f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:02.219980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd440911-6795-4fea-91c5-4e6200f4398d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:02.220684 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd440911-6795-4fea-91c5-4e6200f4398d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:02.220968 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd440911-6795-4fea-91c5-4e6200f4398d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:02.221252 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd440911-6795-4fea-91c5-4e6200f4398d admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:02.221998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd440911-6795-4fea-91c5-4e6200f4398d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:02.223709 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd440911-6795-4fea-91c5-4e6200f4398d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:02.235103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1005/1982] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:02.239381 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd440911-6795-4fea-91c5-4e6200f4398d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:02.250580 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd440911-6795-4fea-91c5-4e6200f4398d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:02.255877 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347861a-4942-4ffe-add0-82d16a8e2c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:02.256633 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347861a-4942-4ffe-add0-82d16a8e2c49 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:02.256925 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347861a-4942-4ffe-add0-82d16a8e2c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:02.257196 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347861a-4942-4ffe-add0-82d16a8e2c49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:02.257960 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347861a-4942-4ffe-add0-82d16a8e2c49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:06:02.258333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347861a-4942-4ffe-add0-82d16a8e2c49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:02.259996 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347861a-4942-4ffe-add0-82d16a8e2c49 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:02.274969 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1006/1983] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:06:02.275967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347861a-4942-4ffe-add0-82d16a8e2c49 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:02.283604 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:02.294777 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347861a-4942-4ffe-add0-82d16a8e2c49 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:02.298111 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 978/1984] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:02.299142 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c61e4c-7f4b-436f-952f-b23b82ebcb02 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:02.314004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753b922-c91d-41b8-8b3f-14c7ed5bce01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:02.315090 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753b922-c91d-41b8-8b3f-14c7ed5bce01 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:02.315443 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753b922-c91d-41b8-8b3f-14c7ed5bce01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:02.315817 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753b922-c91d-41b8-8b3f-14c7ed5bce01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:02.317103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753b922-c91d-41b8-8b3f-14c7ed5bce01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:06:02.317610 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753b922-c91d-41b8-8b3f-14c7ed5bce01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:02.320652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753b922-c91d-41b8-8b3f-14c7ed5bce01 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:02.323477 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c61e4c-7f4b-436f-952f-b23b82ebcb02 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:02.331502 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5bc74-df66-4863-9081-eb79550ddc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:02.333141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5bc74-df66-4863-9081-eb79550ddc9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:02.333582 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5bc74-df66-4863-9081-eb79550ddc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:02.333972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5bc74-df66-4863-9081-eb79550ddc9c admin admin] PATH_INFO: `/v3/projects/eaea742a13bd43ba88339d0d1095dbb3/users/4515a88c5876460f8d70aff9f7ca6875/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:02.343894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1007/1985] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:06:02.345601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753b922-c91d-41b8-8b3f-14c7ed5bce01 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:02.360030 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b5bc74-df66-4863-9081-eb79550ddc9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaea742a13bd43ba88339d0d1095dbb3, user_id=4515a88c5876460f8d70aff9f7ca6875, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:02.362397 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753b922-c91d-41b8-8b3f-14c7ed5bce01 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:02.365182 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b5bc74-df66-4863-9081-eb79550ddc9c admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:02.369995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f9977-e08d-420c-9e02-89ef04e003fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:02.370952 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f9977-e08d-420c-9e02-89ef04e003fd admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:02.371351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f9977-e08d-420c-9e02-89ef04e003fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:02.371815 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f9977-e08d-420c-9e02-89ef04e003fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:02.372683 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f9977-e08d-420c-9e02-89ef04e003fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:02.375803 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f9977-e08d-420c-9e02-89ef04e003fd admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:02.407126 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 979/1986] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:02 2022] PUT /identity/v3/projects/eaea742a13bd43ba88339d0d1095dbb3/users/4515a88c5876460f8d70aff9f7ca6875/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:02.408141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5bc74-df66-4863-9081-eb79550ddc9c admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:02.410212 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-99b5bc74-df66-4863-9081-eb79550ddc9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:02.430517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1008/1987] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:02 2022] POST /identity/v3/projects => generated 375 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:02.430517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f9977-e08d-420c-9e02-89ef04e003fd admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:02.512800 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5bc74-df66-4863-9081-eb79550ddc9c admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:02.513296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f9977-e08d-420c-9e02-89ef04e003fd admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:02.522295 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1659203-2192-41ca-a648-ea50b8300e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:02.523476 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1659203-2192-41ca-a648-ea50b8300e43 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:02.523996 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1659203-2192-41ca-a648-ea50b8300e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:02.524506 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1659203-2192-41ca-a648-ea50b8300e43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:02.525868 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1659203-2192-41ca-a648-ea50b8300e43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:02.529347 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1659203-2192-41ca-a648-ea50b8300e43 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:02.571101 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1009/1988] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:02 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:02.571869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ce7b5-da9c-4cad-b72a-7151d692ac7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:02.572536 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d48479-209c-41db-81dd-72e4bca5d002 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:02.572831 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d48479-209c-41db-81dd-72e4bca5d002 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:02.573069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ce7b5-da9c-4cad-b72a-7151d692ac7c service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:02.573215 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d48479-209c-41db-81dd-72e4bca5d002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:02.573358 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ce7b5-da9c-4cad-b72a-7151d692ac7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:02.573786 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ce7b5-da9c-4cad-b72a-7151d692ac7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:02.601939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-91d48479-209c-41db-81dd-72e4bca5d002 None None] MFA Rules not processed for user `4515a88c5876460f8d70aff9f7ca6875`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:02.603211 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-91d48479-209c-41db-81dd-72e4bca5d002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:02.633234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:02.652819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ce7b5-da9c-4cad-b72a-7151d692ac7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:02.654310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ce7b5-da9c-4cad-b72a-7151d692ac7c service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:02.666816 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 980/1989] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:02 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:02.667662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ce7b5-da9c-4cad-b72a-7151d692ac7c service cinder] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:02.685363 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1010/1990] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:02.685776 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ce7b5-da9c-4cad-b72a-7151d692ac7c service cinder] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:02.686081 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d48479-209c-41db-81dd-72e4bca5d002 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:02.784567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9335b3c-f0d4-425c-a9aa-b451a435bd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:02.785039 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9335b3c-f0d4-425c-a9aa-b451a435bd6c admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:02.785427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9335b3c-f0d4-425c-a9aa-b451a435bd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:02.785803 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9335b3c-f0d4-425c-a9aa-b451a435bd6c admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:02.786764 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9335b3c-f0d4-425c-a9aa-b451a435bd6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:02.789166 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9335b3c-f0d4-425c-a9aa-b451a435bd6c admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:02.819973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:02.820256 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 981/1991] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:02 2022] POST /identity/v3/users => generated 288 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:02.821302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9335b3c-f0d4-425c-a9aa-b451a435bd6c admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:02.835871 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9335b3c-f0d4-425c-a9aa-b451a435bd6c admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:02.841956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34265d-c096-4efb-b404-6b3a3a92057a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:02.842731 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34265d-c096-4efb-b404-6b3a3a92057a admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:02.843996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34265d-c096-4efb-b404-6b3a3a92057a admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:02.844736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34265d-c096-4efb-b404-6b3a3a92057a admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:02.845705 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34265d-c096-4efb-b404-6b3a3a92057a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:02.848609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34265d-c096-4efb-b404-6b3a3a92057a admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:02.877121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d48479-209c-41db-81dd-72e4bca5d002 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:02.891361 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581aba69-fdfa-499e-94c7-1cd42e218e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:02.892020 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581aba69-fdfa-499e-94c7-1cd42e218e93 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:02.896512 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581aba69-fdfa-499e-94c7-1cd42e218e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:02.896864 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581aba69-fdfa-499e-94c7-1cd42e218e93 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:02.907668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 982/1992] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:02 2022] POST /identity/v3/projects => generated 385 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:02.908591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34265d-c096-4efb-b404-6b3a3a92057a admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:02.926154 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581aba69-fdfa-499e-94c7-1cd42e218e93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:02.928231 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581aba69-fdfa-499e-94c7-1cd42e218e93 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:02.969758 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1011/1993] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:02 2022] POST /identity/v3/users => generated 287 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:02.971151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581aba69-fdfa-499e-94c7-1cd42e218e93 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:02.991384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581aba69-fdfa-499e-94c7-1cd42e218e93 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:03.052160 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34265d-c096-4efb-b404-6b3a3a92057a admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:03.059722 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f95d3-ed21-4411-8efd-d8641eb2268e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:03.060865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f95d3-ed21-4411-8efd-d8641eb2268e admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:03.060865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f95d3-ed21-4411-8efd-d8641eb2268e admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:03.061197 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f95d3-ed21-4411-8efd-d8641eb2268e admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:03.062588 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f95d3-ed21-4411-8efd-d8641eb2268e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:03.064425 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f95d3-ed21-4411-8efd-d8641eb2268e admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:03.073620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df400d62-1926-43bf-971e-557d5106015c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:03.075002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df400d62-1926-43bf-971e-557d5106015c admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:03.075544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df400d62-1926-43bf-971e-557d5106015c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:03.076002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df400d62-1926-43bf-971e-557d5106015c admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:03.077040 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df400d62-1926-43bf-971e-557d5106015c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:03.080096 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 983/1994] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:03.081120 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f95d3-ed21-4411-8efd-d8641eb2268e admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:03.083026 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df400d62-1926-43bf-971e-557d5106015c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:03.091895 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f95d3-ed21-4411-8efd-d8641eb2268e admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:03.097906 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf27dd4-b6c5-4f9f-9eec-604434d95778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:03.098916 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf27dd4-b6c5-4f9f-9eec-604434d95778 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:03.099469 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf27dd4-b6c5-4f9f-9eec-604434d95778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:03.099469 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf27dd4-b6c5-4f9f-9eec-604434d95778 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:03.100222 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf27dd4-b6c5-4f9f-9eec-604434d95778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:03.101923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf27dd4-b6c5-4f9f-9eec-604434d95778 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:03.112329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1012/1995] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:02 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:03.113732 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df400d62-1926-43bf-971e-557d5106015c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:03.116377 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 984/1996] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:03.120810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf27dd4-b6c5-4f9f-9eec-604434d95778 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:03.126445 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df400d62-1926-43bf-971e-557d5106015c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:03.131871 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea5837-8d76-493c-8693-ab9ad2fd74ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:03.133153 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea5837-8d76-493c-8693-ab9ad2fd74ed admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:03.133153 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea5837-8d76-493c-8693-ab9ad2fd74ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:03.133153 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea5837-8d76-493c-8693-ab9ad2fd74ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:03.133885 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea5837-8d76-493c-8693-ab9ad2fd74ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:03.135546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea5837-8d76-493c-8693-ab9ad2fd74ed admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:03.135903 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf27dd4-b6c5-4f9f-9eec-604434d95778 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:03.148337 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1c5f5-b4eb-4505-8fe2-881db742e987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:03.149180 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1c5f5-b4eb-4505-8fe2-881db742e987 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:03.149512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1c5f5-b4eb-4505-8fe2-881db742e987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:03.149785 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1c5f5-b4eb-4505-8fe2-881db742e987 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:03.150572 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1c5f5-b4eb-4505-8fe2-881db742e987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:03.152373 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1c5f5-b4eb-4505-8fe2-881db742e987 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:03.153661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1013/1997] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:03.154351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea5837-8d76-493c-8693-ab9ad2fd74ed admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:03.176006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea5837-8d76-493c-8693-ab9ad2fd74ed admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:03.176217 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 985/1998] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:03.176217 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1c5f5-b4eb-4505-8fe2-881db742e987 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:03.182422 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99956e20-7f5a-4ec6-8beb-8319b585121f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:03.183235 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99956e20-7f5a-4ec6-8beb-8319b585121f admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:03.183336 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1c5f5-b4eb-4505-8fe2-881db742e987 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:03.183617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99956e20-7f5a-4ec6-8beb-8319b585121f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:03.183972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99956e20-7f5a-4ec6-8beb-8319b585121f admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:03.184897 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99956e20-7f5a-4ec6-8beb-8319b585121f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:03.187646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99956e20-7f5a-4ec6-8beb-8319b585121f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:03.192616 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ab666-d939-4e05-82c4-684ad614f2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:03.193629 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ab666-d939-4e05-82c4-684ad614f2c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:03.194014 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ab666-d939-4e05-82c4-684ad614f2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:03.194399 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ab666-d939-4e05-82c4-684ad614f2c3 admin admin] PATH_INFO: `/v3/projects/1f2297508a2a4ef0a8cfceedc4fad02f/users/1a0eea4729bb403d9fa6cfac657f72fa/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:03.200895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1014/1999] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:03.201799 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99956e20-7f5a-4ec6-8beb-8319b585121f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:03.213476 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99956e20-7f5a-4ec6-8beb-8319b585121f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:03.217180 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ab666-d939-4e05-82c4-684ad614f2c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f2297508a2a4ef0a8cfceedc4fad02f, user_id=1a0eea4729bb403d9fa6cfac657f72fa, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:03.218959 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ad466-72c2-46e6-ab5d-66829582c6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:03.219795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ad466-72c2-46e6-ab5d-66829582c6e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:03.220086 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ad466-72c2-46e6-ab5d-66829582c6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:03.220364 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ad466-72c2-46e6-ab5d-66829582c6e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:03.220866 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ab666-d939-4e05-82c4-684ad614f2c3 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:03.221241 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ad466-72c2-46e6-ab5d-66829582c6e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:03.222925 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ad466-72c2-46e6-ab5d-66829582c6e2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:03.236353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1015/2000] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:03.246339 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ad466-72c2-46e6-ab5d-66829582c6e2 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:03.246547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 986/2001] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:03 2022] PUT /identity/v3/projects/1f2297508a2a4ef0a8cfceedc4fad02f/users/1a0eea4729bb403d9fa6cfac657f72fa/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:03.246547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ab666-d939-4e05-82c4-684ad614f2c3 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:03.256458 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:03.260439 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ad466-72c2-46e6-ab5d-66829582c6e2 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:03.365487 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ab666-d939-4e05-82c4-684ad614f2c3 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:03.372121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7058503-a8e3-4f1e-854a-b48207442200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:03.372473 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2c502-6260-4b15-b19c-54820c675a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:03.373042 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7058503-a8e3-4f1e-854a-b48207442200 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:03.373387 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7058503-a8e3-4f1e-854a-b48207442200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:03.373553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2c502-6260-4b15-b19c-54820c675a68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:03.373726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7058503-a8e3-4f1e-854a-b48207442200 admin admin] PATH_INFO: `/v3/projects/84ab655e4a5e44da9c4a8b8c696c492c/users/ca07c30dcc6e466d992db225a6410e62/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:03.373883 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2c502-6260-4b15-b19c-54820c675a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:03.374453 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2c502-6260-4b15-b19c-54820c675a68 admin admin] PATH_INFO: `/v3/projects/66edceff07334667985a940cd9cc9bfe/users/89db5667f0c145878eff188e22352660/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:03.392786 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7058503-a8e3-4f1e-854a-b48207442200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84ab655e4a5e44da9c4a8b8c696c492c, user_id=ca07c30dcc6e466d992db225a6410e62, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:03.395678 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7058503-a8e3-4f1e-854a-b48207442200 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:03.396059 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2c502-6260-4b15-b19c-54820c675a68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66edceff07334667985a940cd9cc9bfe, user_id=89db5667f0c145878eff188e22352660, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:03.399328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2c502-6260-4b15-b19c-54820c675a68 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:03.417133 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1016/2002] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:03 2022] PUT /identity/v3/projects/84ab655e4a5e44da9c4a8b8c696c492c/users/ca07c30dcc6e466d992db225a6410e62/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:03.419315 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90881e7-a09e-43f0-bc16-d6cd9b07967c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:03.419949 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90881e7-a09e-43f0-bc16-d6cd9b07967c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:03.420403 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90881e7-a09e-43f0-bc16-d6cd9b07967c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:03.431870 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 987/2003] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:03 2022] PUT /identity/v3/projects/66edceff07334667985a940cd9cc9bfe/users/89db5667f0c145878eff188e22352660/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:03.431870 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2c502-6260-4b15-b19c-54820c675a68 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:03.458557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:03.476191 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d90881e7-a09e-43f0-bc16-d6cd9b07967c None None] MFA Rules not processed for user `1a0eea4729bb403d9fa6cfac657f72fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:03.481533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d90881e7-a09e-43f0-bc16-d6cd9b07967c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:03.542433 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2c502-6260-4b15-b19c-54820c675a68 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:03.549238 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06f9d1-40ec-4fae-bee8-921e6626b1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:03.550299 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06f9d1-40ec-4fae-bee8-921e6626b1cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:03.550662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06f9d1-40ec-4fae-bee8-921e6626b1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:03.551030 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06f9d1-40ec-4fae-bee8-921e6626b1cc admin admin] PATH_INFO: `/v3/users/35c3f499f55444a994cfe41460a6b1e1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:03.552397 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06f9d1-40ec-4fae-bee8-921e6626b1cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=35c3f499f55444a994cfe41460a6b1e1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:03.554431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06f9d1-40ec-4fae-bee8-921e6626b1cc admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:03.586220 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1017/2004] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:03.586220 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f873a56-aa00-4579-b3fa-7cf219270309 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:03.586220 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f873a56-aa00-4579-b3fa-7cf219270309 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:03.586220 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f873a56-aa00-4579-b3fa-7cf219270309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:03.618837 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2e06f9d1-40ec-4fae-bee8-921e6626b1cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35c3f499f55444a994cfe41460a6b1e1'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:03.628938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2e06f9d1-40ec-4fae-bee8-921e6626b1cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35c3f499f55444a994cfe41460a6b1e1'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:03.629591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2e06f9d1-40ec-4fae-bee8-921e6626b1cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35c3f499f55444a994cfe41460a6b1e1'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:03.630717 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2e06f9d1-40ec-4fae-bee8-921e6626b1cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35c3f499f55444a994cfe41460a6b1e1'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:03.635102 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-7f873a56-aa00-4579-b3fa-7cf219270309 None None] MFA Rules not processed for user `ca07c30dcc6e466d992db225a6410e62`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:03.637257 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7f873a56-aa00-4579-b3fa-7cf219270309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:03.641875 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2e06f9d1-40ec-4fae-bee8-921e6626b1cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35c3f499f55444a994cfe41460a6b1e1'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:03.668784 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 988/2005] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:03 2022] DELETE /identity/v3/users/35c3f499f55444a994cfe41460a6b1e1 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:03.668784 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826e52e-5d01-4d6b-a7b1-c8982128b2c1 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:03.668784 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826e52e-5d01-4d6b-a7b1-c8982128b2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:03.668784 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826e52e-5d01-4d6b-a7b1-c8982128b2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:03.675428 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:03.705006 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-1826e52e-5d01-4d6b-a7b1-c8982128b2c1 None None] MFA Rules not processed for user `89db5667f0c145878eff188e22352660`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:03.706682 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-1826e52e-5d01-4d6b-a7b1-c8982128b2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:03.768076 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1018/2006] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:03.768076 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f873a56-aa00-4579-b3fa-7cf219270309 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:03.768076 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7f873a56-aa00-4579-b3fa-7cf219270309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:03.787080 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 989/2007] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:03.788446 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56315e-e484-4245-b68d-3d48c68a0cb7 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:03.788732 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56315e-e484-4245-b68d-3d48c68a0cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:03.789004 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56315e-e484-4245-b68d-3d48c68a0cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:03.810230 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-ee56315e-e484-4245-b68d-3d48c68a0cb7 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:03.811518 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-ee56315e-e484-4245-b68d-3d48c68a0cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:03.842705 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f873a56-aa00-4579-b3fa-7cf219270309 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:03.893674 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 990/2008] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:03.896022 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56315e-e484-4245-b68d-3d48c68a0cb7 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:03.898688 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-ee56315e-e484-4245-b68d-3d48c68a0cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:03.919078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:03.926923 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3cb9b3-b510-4a38-bdd3-ee7b776adf92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:03.928475 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cb9b3-b510-4a38-bdd3-ee7b776adf92 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:03.928851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cb9b3-b510-4a38-bdd3-ee7b776adf92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:03.929216 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cb9b3-b510-4a38-bdd3-ee7b776adf92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:03.931359 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7b3cb9b3-b510-4a38-bdd3-ee7b776adf92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:03.997094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56315e-e484-4245-b68d-3d48c68a0cb7 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:04.006116 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1dbb6-0769-4425-9e99-937eea55d7f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:04.006824 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1dbb6-0769-4425-9e99-937eea55d7f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:04.007209 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1dbb6-0769-4425-9e99-937eea55d7f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:04.007366 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1dbb6-0769-4425-9e99-937eea55d7f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:04.039732 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3cb9b3-b510-4a38-bdd3-ee7b776adf92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:04.043006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3cb9b3-b510-4a38-bdd3-ee7b776adf92 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:04.062320 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1019/2009] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:04.064083 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3cb9b3-b510-4a38-bdd3-ee7b776adf92 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:04.066946 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7b3cb9b3-b510-4a38-bdd3-ee7b776adf92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:04.081254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:04.095387 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3cb9b3-b510-4a38-bdd3-ee7b776adf92 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:04.096490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1dbb6-0769-4425-9e99-937eea55d7f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:04.098214 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1dbb6-0769-4425-9e99-937eea55d7f1 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:04.103671 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221bd9d7-9c7b-4845-b219-cae75a9a1513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:04.104824 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221bd9d7-9c7b-4845-b219-cae75a9a1513 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:04.105362 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221bd9d7-9c7b-4845-b219-cae75a9a1513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:04.105795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221bd9d7-9c7b-4845-b219-cae75a9a1513 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:04.107153 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221bd9d7-9c7b-4845-b219-cae75a9a1513 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:04.111846 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221bd9d7-9c7b-4845-b219-cae75a9a1513 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:04.113515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 991/2010] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:04.115061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1dbb6-0769-4425-9e99-937eea55d7f1 service cinder] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:04.116801 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-bca1dbb6-0769-4425-9e99-937eea55d7f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:04.178632 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1020/2011] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:04 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:04.186413 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221bd9d7-9c7b-4845-b219-cae75a9a1513 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:04.196605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-221bd9d7-9c7b-4845-b219-cae75a9a1513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:04.216971 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1dbb6-0769-4425-9e99-937eea55d7f1 service cinder] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:04.300431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41665d-634e-420c-9288-4285a1c0b389 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:04.302516 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41665d-634e-420c-9288-4285a1c0b389 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:04.302516 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41665d-634e-420c-9288-4285a1c0b389 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:04.303021 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41665d-634e-420c-9288-4285a1c0b389 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:04.314589 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221bd9d7-9c7b-4845-b219-cae75a9a1513 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:04.321245 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e0151-6b08-445d-a126-1dfce4ea05b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:04.322621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e0151-6b08-445d-a126-1dfce4ea05b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:04.322914 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e0151-6b08-445d-a126-1dfce4ea05b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:04.323320 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e0151-6b08-445d-a126-1dfce4ea05b4 admin admin] PATH_INFO: `/v3/users/ca07c30dcc6e466d992db225a6410e62` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:04.324982 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e0151-6b08-445d-a126-1dfce4ea05b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca07c30dcc6e466d992db225a6410e62)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:04.328676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e0151-6b08-445d-a126-1dfce4ea05b4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:04.339836 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41665d-634e-420c-9288-4285a1c0b389 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:04.356219 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41665d-634e-420c-9288-4285a1c0b389 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:04.372987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 992/2012] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:04.382517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41665d-634e-420c-9288-4285a1c0b389 service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:04.402228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41665d-634e-420c-9288-4285a1c0b389 service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:04.408055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171707d3-3667-4c63-ab61-ef6a84e899c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:04.408788 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171707d3-3667-4c63-ab61-ef6a84e899c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:04.409076 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171707d3-3667-4c63-ab61-ef6a84e899c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:04.409317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171707d3-3667-4c63-ab61-ef6a84e899c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:04.410142 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171707d3-3667-4c63-ab61-ef6a84e899c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:04.412096 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171707d3-3667-4c63-ab61-ef6a84e899c9 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:04.430261 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-aa3e0151-6b08-445d-a126-1dfce4ea05b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca07c30dcc6e466d992db225a6410e62'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:04.450912 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-aa3e0151-6b08-445d-a126-1dfce4ea05b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca07c30dcc6e466d992db225a6410e62'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:04.452320 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 993/2013] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:04 2022] POST /identity/v3/users => generated 299 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:04.455253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171707d3-3667-4c63-ab61-ef6a84e899c9 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:04.474740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171707d3-3667-4c63-ab61-ef6a84e899c9 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:04.482441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e6b0c1-f971-4e38-9ac5-14b85a9793ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:04.484241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6b0c1-f971-4e38-9ac5-14b85a9793ab admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:04.484907 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6b0c1-f971-4e38-9ac5-14b85a9793ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:04.485302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6b0c1-f971-4e38-9ac5-14b85a9793ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:04.486395 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e6b0c1-f971-4e38-9ac5-14b85a9793ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:04.488377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-aa3e0151-6b08-445d-a126-1dfce4ea05b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca07c30dcc6e466d992db225a6410e62'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:04.488561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e6b0c1-f971-4e38-9ac5-14b85a9793ab admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:04.493429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-aa3e0151-6b08-445d-a126-1dfce4ea05b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca07c30dcc6e466d992db225a6410e62'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:04.497000 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-aa3e0151-6b08-445d-a126-1dfce4ea05b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca07c30dcc6e466d992db225a6410e62'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:04.503491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 994/2014] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:04.509961 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e6b0c1-f971-4e38-9ac5-14b85a9793ab admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:04.512456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-c7e6b0c1-f971-4e38-9ac5-14b85a9793ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:04.529803 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e6b0c1-f971-4e38-9ac5-14b85a9793ab admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:04.537170 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d75bd-3f4e-430c-b6a1-cb7c1378f146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:04.538602 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d75bd-3f4e-430c-b6a1-cb7c1378f146 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:04.539146 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d75bd-3f4e-430c-b6a1-cb7c1378f146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:04.539661 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d75bd-3f4e-430c-b6a1-cb7c1378f146 admin admin] PATH_INFO: `/v3/projects/f1a7e81034eb439591a666ad30796c38/users/2bf6d92d019642fa8e40a5e1fd66f01c/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:04.540630 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:04.565686 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d75bd-3f4e-430c-b6a1-cb7c1378f146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1a7e81034eb439591a666ad30796c38, user_id=2bf6d92d019642fa8e40a5e1fd66f01c, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:04.570828 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d75bd-3f4e-430c-b6a1-cb7c1378f146 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:04.586978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1021/2015] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:04 2022] DELETE /identity/v3/users/ca07c30dcc6e466d992db225a6410e62 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:04.589726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fea578-f6bb-40ab-83f8-95aeebe19f9c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:04.590273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fea578-f6bb-40ab-83f8-95aeebe19f9c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:04.596861 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fea578-f6bb-40ab-83f8-95aeebe19f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:04.608376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 995/2016] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:04 2022] PUT /identity/v3/projects/f1a7e81034eb439591a666ad30796c38/users/2bf6d92d019642fa8e40a5e1fd66f01c/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:04.611027 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d75bd-3f4e-430c-b6a1-cb7c1378f146 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:04.650543 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-60fea578-f6bb-40ab-83f8-95aeebe19f9c None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:04.653655 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-60fea578-f6bb-40ab-83f8-95aeebe19f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:04.741555 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:04.763270 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d75bd-3f4e-430c-b6a1-cb7c1378f146 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:04.775029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca82f98-6034-450d-a8fa-282539b55f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:04.777834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca82f98-6034-450d-a8fa-282539b55f92 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:04.778530 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca82f98-6034-450d-a8fa-282539b55f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:04.778887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca82f98-6034-450d-a8fa-282539b55f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:04.782781 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca82f98-6034-450d-a8fa-282539b55f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:04.789001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca82f98-6034-450d-a8fa-282539b55f92 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:04.790989 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1022/2017] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:04.806033 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fea578-f6bb-40ab-83f8-95aeebe19f9c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:04.810854 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-60fea578-f6bb-40ab-83f8-95aeebe19f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:04.815741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 996/2018] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:04.818048 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca82f98-6034-450d-a8fa-282539b55f92 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:04.840572 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca82f98-6034-450d-a8fa-282539b55f92 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:04.848103 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15530a-5af1-419b-bede-e6b8644be5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:04.849193 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15530a-5af1-419b-bede-e6b8644be5b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:04.849522 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15530a-5af1-419b-bede-e6b8644be5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:04.849874 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15530a-5af1-419b-bede-e6b8644be5b6 admin admin] PATH_INFO: `/v3/projects/f1a7e81034eb439591a666ad30796c38/users/2bf6d92d019642fa8e40a5e1fd66f01c/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:04.852470 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b15530a-5af1-419b-bede-e6b8644be5b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1a7e81034eb439591a666ad30796c38, user_id=2bf6d92d019642fa8e40a5e1fd66f01c, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:04.855470 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b15530a-5af1-419b-bede-e6b8644be5b6 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:04.868284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 997/2019] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:04 2022] PUT /identity/v3/projects/f1a7e81034eb439591a666ad30796c38/users/2bf6d92d019642fa8e40a5e1fd66f01c/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:06:04.880321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e6eb5-22ac-43b1-8139-5fdd103a91c9 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:04.880638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e6eb5-22ac-43b1-8139-5fdd103a91c9 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:04.880950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e6eb5-22ac-43b1-8139-5fdd103a91c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:04.898219 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fea578-f6bb-40ab-83f8-95aeebe19f9c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:04.917459 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-fe7e6eb5-22ac-43b1-8139-5fdd103a91c9 None None] MFA Rules not processed for user `2bf6d92d019642fa8e40a5e1fd66f01c`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:04.919152 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-fe7e6eb5-22ac-43b1-8139-5fdd103a91c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:05.005034 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0de2ec-59c1-4ff6-92c0-ebb6a53e005a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:05.007512 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0de2ec-59c1-4ff6-92c0-ebb6a53e005a service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:05.008441 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0de2ec-59c1-4ff6-92c0-ebb6a53e005a service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:05.009134 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0de2ec-59c1-4ff6-92c0-ebb6a53e005a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:05.045150 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 998/2020] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:05.046360 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e6eb5-22ac-43b1-8139-5fdd103a91c9 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:05.052257 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-fe7e6eb5-22ac-43b1-8139-5fdd103a91c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:05.106561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0de2ec-59c1-4ff6-92c0-ebb6a53e005a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:05.109383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0de2ec-59c1-4ff6-92c0-ebb6a53e005a service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:05.122352 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1023/2021] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:05.123366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0de2ec-59c1-4ff6-92c0-ebb6a53e005a service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:05.126282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fa0de2ec-59c1-4ff6-92c0-ebb6a53e005a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:05.145776 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e6eb5-22ac-43b1-8139-5fdd103a91c9 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:05.151012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:05.156568 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40e604-f160-4678-97f3-eab20d4ab15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:05.159270 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40e604-f160-4678-97f3-eab20d4ab15a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:05.160183 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40e604-f160-4678-97f3-eab20d4ab15a admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:05.160796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40e604-f160-4678-97f3-eab20d4ab15a admin admin] PATH_INFO: `/v3/projects/92bbac43c7b34d73be60ddcde92ae3fd` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:05.164876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40e604-f160-4678-97f3-eab20d4ab15a admin admin] RBAC: Authorizing `identity:delete_project(project_id=92bbac43c7b34d73be60ddcde92ae3fd)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:05.168384 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40e604-f160-4678-97f3-eab20d4ab15a admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:05.246697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0de2ec-59c1-4ff6-92c0-ebb6a53e005a service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:05.259420 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-fd40e604-f160-4678-97f3-eab20d4ab15a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92bbac43c7b34d73be60ddcde92ae3fd', 'request_id': 'req-fd40e604-f160-4678-97f3-eab20d4ab15a'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:05.272393 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-fd40e604-f160-4678-97f3-eab20d4ab15a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92bbac43c7b34d73be60ddcde92ae3fd', 'request_id': 'req-fd40e604-f160-4678-97f3-eab20d4ab15a'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:05.280452 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-fd40e604-f160-4678-97f3-eab20d4ab15a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92bbac43c7b34d73be60ddcde92ae3fd', 'request_id': 'req-fd40e604-f160-4678-97f3-eab20d4ab15a'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:05.281997 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:05.299845 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-fd40e604-f160-4678-97f3-eab20d4ab15a admin admin] The token cache is being invalidate because project 92bbac43c7b34d73be60ddcde92ae3fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:05.300420 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-fd40e604-f160-4678-97f3-eab20d4ab15a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:05.302060 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-fd40e604-f160-4678-97f3-eab20d4ab15a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:05.304352 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 999/2022] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:05 2022] DELETE /identity/v3/projects/92bbac43c7b34d73be60ddcde92ae3fd => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:05.309265 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40e604-f160-4678-97f3-eab20d4ab15a admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:05.312534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-fd40e604-f160-4678-97f3-eab20d4ab15a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:05.351839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822b163-b528-43db-88e2-2c3af23cf166 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:05.354242 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822b163-b528-43db-88e2-2c3af23cf166 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:05.354513 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822b163-b528-43db-88e2-2c3af23cf166 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:05.355017 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822b163-b528-43db-88e2-2c3af23cf166 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:05.361195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a822b163-b528-43db-88e2-2c3af23cf166 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:05.414590 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40e604-f160-4678-97f3-eab20d4ab15a admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:05.420198 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5415ca6-2549-4675-bfab-4c3f0bffb961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:05.421202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5415ca6-2549-4675-bfab-4c3f0bffb961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:05.421437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5415ca6-2549-4675-bfab-4c3f0bffb961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:05.421716 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5415ca6-2549-4675-bfab-4c3f0bffb961 admin admin] PATH_INFO: `/v3/users/23abea28d5aa4265b355f4980354498f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:05.423091 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5415ca6-2549-4675-bfab-4c3f0bffb961 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23abea28d5aa4265b355f4980354498f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:05.425080 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5415ca6-2549-4675-bfab-4c3f0bffb961 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:05.491374 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822b163-b528-43db-88e2-2c3af23cf166 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:05.493259 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822b163-b528-43db-88e2-2c3af23cf166 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:05.505789 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1024/2023] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:05.517603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d5415ca6-2549-4675-bfab-4c3f0bffb961 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23abea28d5aa4265b355f4980354498f'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:05.529436 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d5415ca6-2549-4675-bfab-4c3f0bffb961 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23abea28d5aa4265b355f4980354498f'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:05.530697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d5415ca6-2549-4675-bfab-4c3f0bffb961 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23abea28d5aa4265b355f4980354498f'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:05.531724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d5415ca6-2549-4675-bfab-4c3f0bffb961 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23abea28d5aa4265b355f4980354498f'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:05.541888 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d5415ca6-2549-4675-bfab-4c3f0bffb961 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23abea28d5aa4265b355f4980354498f'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:05.576501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1000/2024] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:05 2022] DELETE /identity/v3/users/23abea28d5aa4265b355f4980354498f => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:06:05.592010 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822b163-b528-43db-88e2-2c3af23cf166 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:05.594336 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a822b163-b528-43db-88e2-2c3af23cf166 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:05.611479 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822b163-b528-43db-88e2-2c3af23cf166 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:05.697337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b0b78b-76f2-4095-b745-9f1cb4fd2640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:05.698287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0b78b-76f2-4095-b745-9f1cb4fd2640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:05.698566 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0b78b-76f2-4095-b745-9f1cb4fd2640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:05.698842 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0b78b-76f2-4095-b745-9f1cb4fd2640 admin admin] PATH_INFO: `/v3/projects/84ab655e4a5e44da9c4a8b8c696c492c` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:05.700650 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b0b78b-76f2-4095-b745-9f1cb4fd2640 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84ab655e4a5e44da9c4a8b8c696c492c)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:05.705966 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b0b78b-76f2-4095-b745-9f1cb4fd2640 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:05.759733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:05.816202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f0b0b78b-76f2-4095-b745-9f1cb4fd2640 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84ab655e4a5e44da9c4a8b8c696c492c', 'request_id': 'req-f0b0b78b-76f2-4095-b745-9f1cb4fd2640'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:05.837290 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f0b0b78b-76f2-4095-b745-9f1cb4fd2640 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84ab655e4a5e44da9c4a8b8c696c492c', 'request_id': 'req-f0b0b78b-76f2-4095-b745-9f1cb4fd2640'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:05.867495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f0b0b78b-76f2-4095-b745-9f1cb4fd2640 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84ab655e4a5e44da9c4a8b8c696c492c', 'request_id': 'req-f0b0b78b-76f2-4095-b745-9f1cb4fd2640'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:05.889461 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f0b0b78b-76f2-4095-b745-9f1cb4fd2640 admin admin] The token cache is being invalidate because project 84ab655e4a5e44da9c4a8b8c696c492c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:05.892492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f0b0b78b-76f2-4095-b745-9f1cb4fd2640 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:05.894097 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f0b0b78b-76f2-4095-b745-9f1cb4fd2640 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:05.900289 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1025/2025] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:05 2022] DELETE /identity/v3/projects/84ab655e4a5e44da9c4a8b8c696c492c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:05.903106 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02dba2-83cd-4e16-bf19-ee1dbaf1df1c None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:05.903718 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02dba2-83cd-4e16-bf19-ee1dbaf1df1c None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:05.904286 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02dba2-83cd-4e16-bf19-ee1dbaf1df1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:05.987240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-8a02dba2-83cd-4e16-bf19-ee1dbaf1df1c None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:05.990672 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-8a02dba2-83cd-4e16-bf19-ee1dbaf1df1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:06.132736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1001/2026] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:06.135302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aab207-21cc-4b11-8410-ed88a9b4ab4d None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:06.135798 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aab207-21cc-4b11-8410-ed88a9b4ab4d None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:06.136515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aab207-21cc-4b11-8410-ed88a9b4ab4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:06.137960 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b0b78b-76f2-4095-b745-9f1cb4fd2640 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:06.147364 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f0b0b78b-76f2-4095-b745-9f1cb4fd2640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:06.180719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-88aab207-21cc-4b11-8410-ed88a9b4ab4d None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:06.181470 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-88aab207-21cc-4b11-8410-ed88a9b4ab4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:06.198325 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b0b78b-76f2-4095-b745-9f1cb4fd2640 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:06.204214 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1002/2027] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:06.206114 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aab207-21cc-4b11-8410-ed88a9b4ab4d None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:06.227887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f34b4-91fd-4750-93cb-147706d177d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:06.229149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f34b4-91fd-4750-93cb-147706d177d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:06.229602 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f34b4-91fd-4750-93cb-147706d177d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:06.230115 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f34b4-91fd-4750-93cb-147706d177d3 admin admin] PATH_INFO: `/v3/projects/a0b4de8454c44e448b78c0ad666f4b76` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:06.231948 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f34b4-91fd-4750-93cb-147706d177d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0b4de8454c44e448b78c0ad666f4b76)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:06.233371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aab207-21cc-4b11-8410-ed88a9b4ab4d None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:06.235168 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f34b4-91fd-4750-93cb-147706d177d3 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:06.251907 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b8bd0-c66b-456e-ba53-f1cdc979d4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:06.251907 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b8bd0-c66b-456e-ba53-f1cdc979d4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:06.251907 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b8bd0-c66b-456e-ba53-f1cdc979d4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:06.251907 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b8bd0-c66b-456e-ba53-f1cdc979d4a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:06.252256 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b8bd0-c66b-456e-ba53-f1cdc979d4a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:06:06.252256 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b8bd0-c66b-456e-ba53-f1cdc979d4a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:06.255356 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b8bd0-c66b-456e-ba53-f1cdc979d4a7 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:06.260638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:06.290470 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1003/2028] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:06:06.297404 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b8bd0-c66b-456e-ba53-f1cdc979d4a7 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:06.310830 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b8bd0-c66b-456e-ba53-f1cdc979d4a7 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:06.331372 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a64f34b4-91fd-4750-93cb-147706d177d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0b4de8454c44e448b78c0ad666f4b76', 'request_id': 'req-a64f34b4-91fd-4750-93cb-147706d177d3'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:06.338976 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a64f34b4-91fd-4750-93cb-147706d177d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0b4de8454c44e448b78c0ad666f4b76', 'request_id': 'req-a64f34b4-91fd-4750-93cb-147706d177d3'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:06.347452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a64f34b4-91fd-4750-93cb-147706d177d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0b4de8454c44e448b78c0ad666f4b76', 'request_id': 'req-a64f34b4-91fd-4750-93cb-147706d177d3'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:06.370291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a64f34b4-91fd-4750-93cb-147706d177d3 admin admin] The token cache is being invalidate because project a0b4de8454c44e448b78c0ad666f4b76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:06.370960 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a64f34b4-91fd-4750-93cb-147706d177d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:06.371880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a64f34b4-91fd-4750-93cb-147706d177d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:06.375455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1026/2029] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:06 2022] DELETE /identity/v3/projects/a0b4de8454c44e448b78c0ad666f4b76 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:06.382818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89beaf-0b41-4127-bc75-e9fcf1751e58 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:06.383272 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89beaf-0b41-4127-bc75-e9fcf1751e58 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:06.383746 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89beaf-0b41-4127-bc75-e9fcf1751e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:06.400604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b7e4c-8131-4818-afc9-cc329df0bba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:06.401301 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b7e4c-8131-4818-afc9-cc329df0bba4 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:06.401590 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b7e4c-8131-4818-afc9-cc329df0bba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:06.401906 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b7e4c-8131-4818-afc9-cc329df0bba4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:06.403397 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b7e4c-8131-4818-afc9-cc329df0bba4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:06.406539 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b7e4c-8131-4818-afc9-cc329df0bba4 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:06.409450 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2d89beaf-0b41-4127-bc75-e9fcf1751e58 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:06.410913 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2d89beaf-0b41-4127-bc75-e9fcf1751e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:06.421906 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1027/2030] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:06.431175 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fa14d-d65a-43a6-8d05-3425d786dfae None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:06.431774 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fa14d-d65a-43a6-8d05-3425d786dfae None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:06.432405 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fa14d-d65a-43a6-8d05-3425d786dfae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:06.439693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1004/2031] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:06 2022] POST /identity/v3/projects => generated 365 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:06:06.446167 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b7e4c-8131-4818-afc9-cc329df0bba4 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:06.448053 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-755b7e4c-8131-4818-afc9-cc329df0bba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:06.481840 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-920fa14d-d65a-43a6-8d05-3425d786dfae None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:06.485115 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-920fa14d-d65a-43a6-8d05-3425d786dfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:06.554511 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b7e4c-8131-4818-afc9-cc329df0bba4 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:06.560737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb24be-0420-4794-908c-5f3b36f55a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:06.561287 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb24be-0420-4794-908c-5f3b36f55a82 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:06.561617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb24be-0420-4794-908c-5f3b36f55a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:06.561892 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb24be-0420-4794-908c-5f3b36f55a82 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:06.562645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb24be-0420-4794-908c-5f3b36f55a82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:06.564751 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb24be-0420-4794-908c-5f3b36f55a82 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:06.588936 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1005/2032] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:06 2022] POST /identity/v3/users => generated 283 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:06.595608 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb24be-0420-4794-908c-5f3b36f55a82 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:06.608316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb24be-0420-4794-908c-5f3b36f55a82 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:06.616039 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78d25d-3a4b-42d9-957b-e5a65006f7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:06.616515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78d25d-3a4b-42d9-957b-e5a65006f7e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:06.616799 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78d25d-3a4b-42d9-957b-e5a65006f7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:06.617283 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78d25d-3a4b-42d9-957b-e5a65006f7e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:06.617811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78d25d-3a4b-42d9-957b-e5a65006f7e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:06.618423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1028/2033] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:06.623223 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fa14d-d65a-43a6-8d05-3425d786dfae None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:06.623454 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78d25d-3a4b-42d9-957b-e5a65006f7e3 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:06.635461 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fa14d-d65a-43a6-8d05-3425d786dfae None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:06.643931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3edb364-b3d4-4de9-bf8e-39f2f7c025b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:06.644766 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1006/2034] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:06.646854 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edb364-b3d4-4de9-bf8e-39f2f7c025b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:06.647205 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edb364-b3d4-4de9-bf8e-39f2f7c025b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:06.647611 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edb364-b3d4-4de9-bf8e-39f2f7c025b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:06.648635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3edb364-b3d4-4de9-bf8e-39f2f7c025b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:06:06.648907 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78d25d-3a4b-42d9-957b-e5a65006f7e3 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:06.649006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3edb364-b3d4-4de9-bf8e-39f2f7c025b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:06.650950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3edb364-b3d4-4de9-bf8e-39f2f7c025b8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:06.660960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78d25d-3a4b-42d9-957b-e5a65006f7e3 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:06.666774 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26577054-f516-4549-b5f4-9af9fb03dc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:06.667481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26577054-f516-4549-b5f4-9af9fb03dc1b admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:06.667777 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26577054-f516-4549-b5f4-9af9fb03dc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:06.668053 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26577054-f516-4549-b5f4-9af9fb03dc1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:06.668792 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26577054-f516-4549-b5f4-9af9fb03dc1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:06.670479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26577054-f516-4549-b5f4-9af9fb03dc1b admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:06.678345 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1029/2035] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:06:06.683561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3edb364-b3d4-4de9-bf8e-39f2f7c025b8 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:06.686755 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1007/2036] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:06:06.691657 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26577054-f516-4549-b5f4-9af9fb03dc1b admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:06.698067 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3edb364-b3d4-4de9-bf8e-39f2f7c025b8 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:06.702294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26577054-f516-4549-b5f4-9af9fb03dc1b admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:06.704005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d639ec-c867-41f7-8f8c-1d0cdb9601f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:06.704993 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d639ec-c867-41f7-8f8c-1d0cdb9601f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:06.705287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d639ec-c867-41f7-8f8c-1d0cdb9601f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:06.705562 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d639ec-c867-41f7-8f8c-1d0cdb9601f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:06.708405 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d0d05-e57f-4b5b-8d11-fac4fa7cbca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:06.709549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d0d05-e57f-4b5b-8d11-fac4fa7cbca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:06.709955 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d0d05-e57f-4b5b-8d11-fac4fa7cbca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:06.710357 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d0d05-e57f-4b5b-8d11-fac4fa7cbca2 admin admin] PATH_INFO: `/v3/projects/a08c291180e4486eaabb9c1e62905ab0/users/372e7144b26748fea5ad297493142cc2/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:06.710980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d639ec-c867-41f7-8f8c-1d0cdb9601f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:06.712977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d639ec-c867-41f7-8f8c-1d0cdb9601f5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:06.730350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d0d05-e57f-4b5b-8d11-fac4fa7cbca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a08c291180e4486eaabb9c1e62905ab0, user_id=372e7144b26748fea5ad297493142cc2, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:06.730941 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:06.733482 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d0d05-e57f-4b5b-8d11-fac4fa7cbca2 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:06.758665 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1030/2037] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:06 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:06:06.758867 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1008/2038] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:06 2022] PUT /identity/v3/projects/a08c291180e4486eaabb9c1e62905ab0/users/372e7144b26748fea5ad297493142cc2/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:06.759863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d639ec-c867-41f7-8f8c-1d0cdb9601f5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:06.774561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812b6ee-3094-44b6-90e8-4fc766d11595 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:06.774995 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812b6ee-3094-44b6-90e8-4fc766d11595 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:06.775383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812b6ee-3094-44b6-90e8-4fc766d11595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:06.813441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-b812b6ee-3094-44b6-90e8-4fc766d11595 None None] MFA Rules not processed for user `372e7144b26748fea5ad297493142cc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:06.814759 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-b812b6ee-3094-44b6-90e8-4fc766d11595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:06.825435 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:06.830862 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d639ec-c867-41f7-8f8c-1d0cdb9601f5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:06.836375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7197a2c-65c7-4930-a60f-3b093a24d24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:06.836790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7197a2c-65c7-4930-a60f-3b093a24d24b admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:06.837240 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7197a2c-65c7-4930-a60f-3b093a24d24b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:06.837519 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7197a2c-65c7-4930-a60f-3b093a24d24b admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:06.838280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7197a2c-65c7-4930-a60f-3b093a24d24b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:06.840279 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7197a2c-65c7-4930-a60f-3b093a24d24b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:06.864711 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1031/2039] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:06 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:06.870555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7197a2c-65c7-4930-a60f-3b093a24d24b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:06.881911 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7197a2c-65c7-4930-a60f-3b093a24d24b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:06.886925 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5d6f1-c201-422e-a614-08d620175614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:06.887829 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5d6f1-c201-422e-a614-08d620175614 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:06.888022 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5d6f1-c201-422e-a614-08d620175614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:06.888485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5d6f1-c201-422e-a614-08d620175614 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:06.889258 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5d6f1-c201-422e-a614-08d620175614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:06.891266 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5d6f1-c201-422e-a614-08d620175614 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:06.904391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1009/2040] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:06:06.908471 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1032/2041] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:06:06.913311 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812b6ee-3094-44b6-90e8-4fc766d11595 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:06.916836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5d6f1-c201-422e-a614-08d620175614 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:06.918827 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bed5d6f1-c201-422e-a614-08d620175614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:06.934142 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812b6ee-3094-44b6-90e8-4fc766d11595 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:06.940333 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f8b60-a8c0-4604-9f4b-ca1fbc230d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:06.941188 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f8b60-a8c0-4604-9f4b-ca1fbc230d76 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:06.941668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f8b60-a8c0-4604-9f4b-ca1fbc230d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:06.942073 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f8b60-a8c0-4604-9f4b-ca1fbc230d76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:06.943040 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f8b60-a8c0-4604-9f4b-ca1fbc230d76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:06.945138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f8b60-a8c0-4604-9f4b-ca1fbc230d76 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:06.957577 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1010/2042] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:06.962078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f8b60-a8c0-4604-9f4b-ca1fbc230d76 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:06.977867 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f8b60-a8c0-4604-9f4b-ca1fbc230d76 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:06.988300 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036aecb-5d67-4c88-8c65-c2b2e2ea55e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:06.989821 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036aecb-5d67-4c88-8c65-c2b2e2ea55e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:06.990314 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036aecb-5d67-4c88-8c65-c2b2e2ea55e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:06.990863 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036aecb-5d67-4c88-8c65-c2b2e2ea55e3 admin admin] PATH_INFO: `/v3/projects/8a1a8e5e70424cb0aa6937d7de6956dc/users/730beefdf51a4c13927b730da3030f7b/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:07.000923 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5d6f1-c201-422e-a614-08d620175614 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:07.005036 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03cf847-0177-4613-9856-5cc83fbc55bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:07.005850 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03cf847-0177-4613-9856-5cc83fbc55bf service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:07.006142 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03cf847-0177-4613-9856-5cc83fbc55bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:07.006491 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03cf847-0177-4613-9856-5cc83fbc55bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:07.015989 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03cf847-0177-4613-9856-5cc83fbc55bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:07.017483 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03cf847-0177-4613-9856-5cc83fbc55bf service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:07.019339 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036aecb-5d67-4c88-8c65-c2b2e2ea55e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a1a8e5e70424cb0aa6937d7de6956dc, user_id=730beefdf51a4c13927b730da3030f7b, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:07.027119 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036aecb-5d67-4c88-8c65-c2b2e2ea55e3 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:07.034505 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1033/2043] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:06 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:07.049060 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1011/2044] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:06 2022] PUT /identity/v3/projects/8a1a8e5e70424cb0aa6937d7de6956dc/users/730beefdf51a4c13927b730da3030f7b/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:07.061067 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ad680-21d9-48aa-bc5c-b18bf0ad94d1 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:07.061364 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ad680-21d9-48aa-bc5c-b18bf0ad94d1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:07.061751 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ad680-21d9-48aa-bc5c-b18bf0ad94d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:07.097153 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0c4ad680-21d9-48aa-bc5c-b18bf0ad94d1 None None] MFA Rules not processed for user `730beefdf51a4c13927b730da3030f7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:07.098505 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0c4ad680-21d9-48aa-bc5c-b18bf0ad94d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:07.177521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:07.182974 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1034/2045] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:07.188734 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036aecb-5d67-4c88-8c65-c2b2e2ea55e3 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:07.287725 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036aecb-5d67-4c88-8c65-c2b2e2ea55e3 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:07.292764 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def59696-1c0c-4217-b845-f47c9add87d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:07.293451 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def59696-1c0c-4217-b845-f47c9add87d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:07.293737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def59696-1c0c-4217-b845-f47c9add87d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:07.294025 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def59696-1c0c-4217-b845-f47c9add87d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:07.294770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def59696-1c0c-4217-b845-f47c9add87d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:07.296797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def59696-1c0c-4217-b845-f47c9add87d7 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:07.329498 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1012/2046] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:07 2022] POST /identity/v3/projects => generated 387 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:07.340134 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ad680-21d9-48aa-bc5c-b18bf0ad94d1 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:07.355165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ad680-21d9-48aa-bc5c-b18bf0ad94d1 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:07.398211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def59696-1c0c-4217-b845-f47c9add87d7 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:07.400411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-def59696-1c0c-4217-b845-f47c9add87d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:07.451853 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c8e02-4e7f-4ede-9070-0e2b89313fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:07.454027 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c8e02-4e7f-4ede-9070-0e2b89313fd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:07.454629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c8e02-4e7f-4ede-9070-0e2b89313fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:07.455743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c8e02-4e7f-4ede-9070-0e2b89313fd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:07.457368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c8e02-4e7f-4ede-9070-0e2b89313fd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:07.461903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c8e02-4e7f-4ede-9070-0e2b89313fd0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:07.495541 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def59696-1c0c-4217-b845-f47c9add87d7 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:07.500963 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af2e3a-5b12-405a-ac4c-3d672b4cb978 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:07.501646 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af2e3a-5b12-405a-ac4c-3d672b4cb978 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:07.501928 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af2e3a-5b12-405a-ac4c-3d672b4cb978 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:07.502222 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af2e3a-5b12-405a-ac4c-3d672b4cb978 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:07.504334 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-50af2e3a-5b12-405a-ac4c-3d672b4cb978 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:07.506375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1035/2047] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:07 2022] POST /identity/v3/users => generated 300 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:07.515432 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c8e02-4e7f-4ede-9070-0e2b89313fd0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:07.526932 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c8e02-4e7f-4ede-9070-0e2b89313fd0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:07.534066 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:07.535603 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6db022-c147-456f-a9ad-e4e36ea31714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:07.536281 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6db022-c147-456f-a9ad-e4e36ea31714 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:07.536816 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6db022-c147-456f-a9ad-e4e36ea31714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:07.537025 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6db022-c147-456f-a9ad-e4e36ea31714 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:07.537814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6db022-c147-456f-a9ad-e4e36ea31714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:07.540376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6db022-c147-456f-a9ad-e4e36ea31714 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:07.554636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1036/2048] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:06:07.562519 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6db022-c147-456f-a9ad-e4e36ea31714 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:07.576448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6db022-c147-456f-a9ad-e4e36ea31714 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:07.581846 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb34e9-4e26-4fc3-a471-27878b1b8a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:07.582984 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb34e9-4e26-4fc3-a471-27878b1b8a1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:07.583279 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb34e9-4e26-4fc3-a471-27878b1b8a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:07.583561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb34e9-4e26-4fc3-a471-27878b1b8a1e admin admin] PATH_INFO: `/v3/projects/bb3ff296fd39497fafc3616096f2232e/users/c98df9c713134cff93847cae17e69c93/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:07.607061 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb34e9-4e26-4fc3-a471-27878b1b8a1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb3ff296fd39497fafc3616096f2232e, user_id=c98df9c713134cff93847cae17e69c93, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:07.610789 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb34e9-4e26-4fc3-a471-27878b1b8a1e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:07.617046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af2e3a-5b12-405a-ac4c-3d672b4cb978 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:07.618759 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af2e3a-5b12-405a-ac4c-3d672b4cb978 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:07.628551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1037/2049] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:07 2022] PUT /identity/v3/projects/bb3ff296fd39497fafc3616096f2232e/users/c98df9c713134cff93847cae17e69c93/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:07.630529 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1013/2050] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:06:07.634302 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb34e9-4e26-4fc3-a471-27878b1b8a1e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:07.725832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb34e9-4e26-4fc3-a471-27878b1b8a1e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:07.730774 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7417e9-694e-486e-a1f0-94e167339ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:07.731468 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7417e9-694e-486e-a1f0-94e167339ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:07.731758 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7417e9-694e-486e-a1f0-94e167339ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:07.732046 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7417e9-694e-486e-a1f0-94e167339ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:07.732782 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7417e9-694e-486e-a1f0-94e167339ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:07.735211 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7417e9-694e-486e-a1f0-94e167339ef8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:07.747717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1038/2051] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:06:07.752325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af2e3a-5b12-405a-ac4c-3d672b4cb978 service cinder] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:07.773383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af2e3a-5b12-405a-ac4c-3d672b4cb978 service cinder] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:07.784080 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034ce8a-d3d9-4a7a-8314-fe391c35e01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:07.784617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034ce8a-d3d9-4a7a-8314-fe391c35e01e admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:07.785098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034ce8a-d3d9-4a7a-8314-fe391c35e01e admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:07.785621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034ce8a-d3d9-4a7a-8314-fe391c35e01e admin admin] PATH_INFO: `/v3/projects/bb3ff296fd39497fafc3616096f2232e/users/c98df9c713134cff93847cae17e69c93/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:07.789155 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034ce8a-d3d9-4a7a-8314-fe391c35e01e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb3ff296fd39497fafc3616096f2232e, user_id=c98df9c713134cff93847cae17e69c93, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:07.795075 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034ce8a-d3d9-4a7a-8314-fe391c35e01e admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:07.819315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1014/2052] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:07 2022] PUT /identity/v3/projects/bb3ff296fd39497fafc3616096f2232e/users/c98df9c713134cff93847cae17e69c93/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:07.838171 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607b9df-9b69-481d-8edd-e6914ba165ab None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:07.838967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607b9df-9b69-481d-8edd-e6914ba165ab None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:07.839129 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607b9df-9b69-481d-8edd-e6914ba165ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:07.877758 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2607b9df-9b69-481d-8edd-e6914ba165ab None None] MFA Rules not processed for user `c98df9c713134cff93847cae17e69c93`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:07.879061 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2607b9df-9b69-481d-8edd-e6914ba165ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:07.955298 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:08.017825 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1039/2053] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:08.033235 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034ce8a-d3d9-4a7a-8314-fe391c35e01e admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:08.109352 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034ce8a-d3d9-4a7a-8314-fe391c35e01e admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:08.113667 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eddf2b3-93ce-4ae9-b005-2c33f44e1940 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:08.114611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddf2b3-93ce-4ae9-b005-2c33f44e1940 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:08.115003 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddf2b3-93ce-4ae9-b005-2c33f44e1940 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:08.115381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddf2b3-93ce-4ae9-b005-2c33f44e1940 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:08.126911 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eddf2b3-93ce-4ae9-b005-2c33f44e1940 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:08.128530 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eddf2b3-93ce-4ae9-b005-2c33f44e1940 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:08.147333 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1015/2054] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:08.226396 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607b9df-9b69-481d-8edd-e6914ba165ab None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:08.302252 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607b9df-9b69-481d-8edd-e6914ba165ab None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:08.306788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e970d3-f43f-416b-bedd-548151c40387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:08.307701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e970d3-f43f-416b-bedd-548151c40387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:08.307980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e970d3-f43f-416b-bedd-548151c40387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:08.308252 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e970d3-f43f-416b-bedd-548151c40387 admin admin] PATH_INFO: `/v3/users/730beefdf51a4c13927b730da3030f7b` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:08.309548 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e970d3-f43f-416b-bedd-548151c40387 admin admin] RBAC: Authorizing `identity:delete_user(user_id=730beefdf51a4c13927b730da3030f7b)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:08.312032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e970d3-f43f-416b-bedd-548151c40387 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:08.370366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-81e970d3-f43f-416b-bedd-548151c40387 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '730beefdf51a4c13927b730da3030f7b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:08.380551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-81e970d3-f43f-416b-bedd-548151c40387 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '730beefdf51a4c13927b730da3030f7b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:08.403366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-81e970d3-f43f-416b-bedd-548151c40387 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '730beefdf51a4c13927b730da3030f7b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:08.404167 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-81e970d3-f43f-416b-bedd-548151c40387 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '730beefdf51a4c13927b730da3030f7b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:08.404784 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-81e970d3-f43f-416b-bedd-548151c40387 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '730beefdf51a4c13927b730da3030f7b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:08.418983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1040/2055] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:08 2022] DELETE /identity/v3/users/730beefdf51a4c13927b730da3030f7b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:08.424121 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724b205-2f2c-4355-94ce-28d5a32a54a2 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:08.424507 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724b205-2f2c-4355-94ce-28d5a32a54a2 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:08.424828 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724b205-2f2c-4355-94ce-28d5a32a54a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:08.444846 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-b724b205-2f2c-4355-94ce-28d5a32a54a2 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:08.446281 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-b724b205-2f2c-4355-94ce-28d5a32a54a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:08.515834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1016/2056] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:06:08.529081 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e970d3-f43f-416b-bedd-548151c40387 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:08.531259 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-81e970d3-f43f-416b-bedd-548151c40387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:08.646956 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e970d3-f43f-416b-bedd-548151c40387 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:08.653717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75f72a-b787-4eb1-9212-affe348fb36b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:08.655192 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75f72a-b787-4eb1-9212-affe348fb36b service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:08.656015 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75f72a-b787-4eb1-9212-affe348fb36b service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:08.656588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75f72a-b787-4eb1-9212-affe348fb36b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:08.675407 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c75f72a-b787-4eb1-9212-affe348fb36b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:08.680551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c75f72a-b787-4eb1-9212-affe348fb36b service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:08.690569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:08.700179 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1041/2057] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:09.047572 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724b205-2f2c-4355-94ce-28d5a32a54a2 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:09.049958 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-b724b205-2f2c-4355-94ce-28d5a32a54a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:09.071031 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724b205-2f2c-4355-94ce-28d5a32a54a2 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:09.082647 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7836a-05a8-4b3a-a370-455f690c2289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:09.084396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7836a-05a8-4b3a-a370-455f690c2289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:09.084937 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7836a-05a8-4b3a-a370-455f690c2289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:09.085448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7836a-05a8-4b3a-a370-455f690c2289 admin admin] PATH_INFO: `/v3/projects/8a1a8e5e70424cb0aa6937d7de6956dc` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:09.087809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7836a-05a8-4b3a-a370-455f690c2289 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a1a8e5e70424cb0aa6937d7de6956dc)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:09.091630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7836a-05a8-4b3a-a370-455f690c2289 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:09.155801 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:09.165488 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-11d7836a-05a8-4b3a-a370-455f690c2289 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a1a8e5e70424cb0aa6937d7de6956dc', 'request_id': 'req-11d7836a-05a8-4b3a-a370-455f690c2289'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:09.175738 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-11d7836a-05a8-4b3a-a370-455f690c2289 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a1a8e5e70424cb0aa6937d7de6956dc', 'request_id': 'req-11d7836a-05a8-4b3a-a370-455f690c2289'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:09.183537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-11d7836a-05a8-4b3a-a370-455f690c2289 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a1a8e5e70424cb0aa6937d7de6956dc', 'request_id': 'req-11d7836a-05a8-4b3a-a370-455f690c2289'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:09.202016 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-11d7836a-05a8-4b3a-a370-455f690c2289 admin admin] The token cache is being invalidate because project 8a1a8e5e70424cb0aa6937d7de6956dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:09.202477 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-11d7836a-05a8-4b3a-a370-455f690c2289 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:09.203899 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-11d7836a-05a8-4b3a-a370-455f690c2289 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:09.205679 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1017/2058] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:09 2022] DELETE /identity/v3/projects/8a1a8e5e70424cb0aa6937d7de6956dc => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:09.210730 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75f72a-b787-4eb1-9212-affe348fb36b service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:09.223803 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75f72a-b787-4eb1-9212-affe348fb36b service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:09.305509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74070e-6b53-4ebb-813b-e4a7ee5e74f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:09.306675 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74070e-6b53-4ebb-813b-e4a7ee5e74f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:09.306675 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74070e-6b53-4ebb-813b-e4a7ee5e74f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:09.306977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74070e-6b53-4ebb-813b-e4a7ee5e74f2 admin admin] PATH_INFO: `/v3/users/c98df9c713134cff93847cae17e69c93` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:09.308237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab74070e-6b53-4ebb-813b-e4a7ee5e74f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c98df9c713134cff93847cae17e69c93)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:09.311084 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab74070e-6b53-4ebb-813b-e4a7ee5e74f2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:09.375574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ab74070e-6b53-4ebb-813b-e4a7ee5e74f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c98df9c713134cff93847cae17e69c93'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:09.388810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ab74070e-6b53-4ebb-813b-e4a7ee5e74f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c98df9c713134cff93847cae17e69c93'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:09.414008 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ab74070e-6b53-4ebb-813b-e4a7ee5e74f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c98df9c713134cff93847cae17e69c93'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:09.415763 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ab74070e-6b53-4ebb-813b-e4a7ee5e74f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c98df9c713134cff93847cae17e69c93'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:09.416960 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ab74070e-6b53-4ebb-813b-e4a7ee5e74f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c98df9c713134cff93847cae17e69c93'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:09.438276 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1042/2059] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:09 2022] DELETE /identity/v3/users/c98df9c713134cff93847cae17e69c93 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:06:09.867034 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7836a-05a8-4b3a-a370-455f690c2289 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:09.868862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-11d7836a-05a8-4b3a-a370-455f690c2289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:09.878287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74070e-6b53-4ebb-813b-e4a7ee5e74f2 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:09.880819 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ab74070e-6b53-4ebb-813b-e4a7ee5e74f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:09.920752 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7836a-05a8-4b3a-a370-455f690c2289 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:10.020281 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74070e-6b53-4ebb-813b-e4a7ee5e74f2 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:10.020771 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2daa1a-f149-45bd-9f35-e4fdb747062e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:10.022772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2daa1a-f149-45bd-9f35-e4fdb747062e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:10.023288 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2daa1a-f149-45bd-9f35-e4fdb747062e admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:10.023833 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2daa1a-f149-45bd-9f35-e4fdb747062e admin admin] PATH_INFO: `/v3/projects/bb3ff296fd39497fafc3616096f2232e` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:10.024418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7912e62-943d-485a-91ef-b6672c4a6077 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:10.025162 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7912e62-943d-485a-91ef-b6672c4a6077 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:10.025572 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7912e62-943d-485a-91ef-b6672c4a6077 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:10.025764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7912e62-943d-485a-91ef-b6672c4a6077 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:10.026467 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2daa1a-f149-45bd-9f35-e4fdb747062e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb3ff296fd39497fafc3616096f2232e)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:10.028508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a7912e62-943d-485a-91ef-b6672c4a6077 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:10.032441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2daa1a-f149-45bd-9f35-e4fdb747062e admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:10.070469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:10.127247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7912e62-943d-485a-91ef-b6672c4a6077 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:10.130010 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7912e62-943d-485a-91ef-b6672c4a6077 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:10.130336 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-0e2daa1a-f149-45bd-9f35-e4fdb747062e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb3ff296fd39497fafc3616096f2232e', 'request_id': 'req-0e2daa1a-f149-45bd-9f35-e4fdb747062e'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:10.145480 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-0e2daa1a-f149-45bd-9f35-e4fdb747062e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb3ff296fd39497fafc3616096f2232e', 'request_id': 'req-0e2daa1a-f149-45bd-9f35-e4fdb747062e'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:10.150166 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1043/2060] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:09 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:06:10.153458 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-0e2daa1a-f149-45bd-9f35-e4fdb747062e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb3ff296fd39497fafc3616096f2232e', 'request_id': 'req-0e2daa1a-f149-45bd-9f35-e4fdb747062e'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:10.173460 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-0e2daa1a-f149-45bd-9f35-e4fdb747062e admin admin] The token cache is being invalidate because project bb3ff296fd39497fafc3616096f2232e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:10.173951 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-0e2daa1a-f149-45bd-9f35-e4fdb747062e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:10.175095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-0e2daa1a-f149-45bd-9f35-e4fdb747062e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:10.178107 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1018/2061] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:09 2022] DELETE /identity/v3/projects/bb3ff296fd39497fafc3616096f2232e => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:10.184424 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffff5e-9539-4212-8cea-640bf75f6bd4 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:10.184878 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffff5e-9539-4212-8cea-640bf75f6bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:10.185334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffff5e-9539-4212-8cea-640bf75f6bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:10.211354 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c4ffff5e-9539-4212-8cea-640bf75f6bd4 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:10.212786 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c4ffff5e-9539-4212-8cea-640bf75f6bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:10.326407 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1044/2062] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:10.332943 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34797a9-52e3-4a18-a59a-6062343df830 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:10.333963 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34797a9-52e3-4a18-a59a-6062343df830 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:10.334520 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34797a9-52e3-4a18-a59a-6062343df830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:10.370059 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-f34797a9-52e3-4a18-a59a-6062343df830 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:10.372230 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f34797a9-52e3-4a18-a59a-6062343df830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:10.384123 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1019/2063] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:10.388732 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffff5e-9539-4212-8cea-640bf75f6bd4 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:10.400084 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffff5e-9539-4212-8cea-640bf75f6bd4 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:10.406123 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5acb0-197a-47a4-8c01-76e0823e7b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:10.406841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5acb0-197a-47a4-8c01-76e0823e7b0a admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:10.407107 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5acb0-197a-47a4-8c01-76e0823e7b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:10.407384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5acb0-197a-47a4-8c01-76e0823e7b0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:10.408200 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf5acb0-197a-47a4-8c01-76e0823e7b0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:06:10.408425 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf5acb0-197a-47a4-8c01-76e0823e7b0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:10.410064 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf5acb0-197a-47a4-8c01-76e0823e7b0a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:10.424196 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1045/2064] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:06:10.428610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34797a9-52e3-4a18-a59a-6062343df830 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:10.439524 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34797a9-52e3-4a18-a59a-6062343df830 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:10.447325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155dd18-3f91-4643-87a4-820f498088de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:10.447746 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155dd18-3f91-4643-87a4-820f498088de admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:10.448040 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155dd18-3f91-4643-87a4-820f498088de admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:10.448387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155dd18-3f91-4643-87a4-820f498088de admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:10.449102 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155dd18-3f91-4643-87a4-820f498088de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:10.451134 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155dd18-3f91-4643-87a4-820f498088de admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:10.500793 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1020/2065] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:10 2022] POST /identity/v3/projects => generated 381 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:10.507501 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5acb0-197a-47a4-8c01-76e0823e7b0a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:10.535677 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5acb0-197a-47a4-8c01-76e0823e7b0a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:10.657823 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023e3ea-d92e-4164-9f78-871145eaabff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:10.658339 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023e3ea-d92e-4164-9f78-871145eaabff admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:10.658836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023e3ea-d92e-4164-9f78-871145eaabff admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:10.659353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023e3ea-d92e-4164-9f78-871145eaabff admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:10.660678 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023e3ea-d92e-4164-9f78-871145eaabff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:10.664589 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023e3ea-d92e-4164-9f78-871145eaabff admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:10.693938 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1046/2066] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:10 2022] POST /identity/v3/users => generated 291 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:10.698958 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155dd18-3f91-4643-87a4-820f498088de admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:10.718568 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155dd18-3f91-4643-87a4-820f498088de admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:10.728439 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7868de-4e37-4191-a59e-b8ffcd5b68f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:10.729406 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7868de-4e37-4191-a59e-b8ffcd5b68f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:10.729873 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7868de-4e37-4191-a59e-b8ffcd5b68f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:10.730363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7868de-4e37-4191-a59e-b8ffcd5b68f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:10.731490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7868de-4e37-4191-a59e-b8ffcd5b68f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:10.734537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7868de-4e37-4191-a59e-b8ffcd5b68f6 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:10.753435 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1021/2067] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:10.759367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023e3ea-d92e-4164-9f78-871145eaabff admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:10.771689 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023e3ea-d92e-4164-9f78-871145eaabff admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:10.776850 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772533d-0e80-427a-9413-c5231b3364f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:10.777364 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772533d-0e80-427a-9413-c5231b3364f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:10.777642 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772533d-0e80-427a-9413-c5231b3364f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:10.777918 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772533d-0e80-427a-9413-c5231b3364f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:10.778708 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2772533d-0e80-427a-9413-c5231b3364f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:10.780669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2772533d-0e80-427a-9413-c5231b3364f0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:10.792293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1047/2068] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:06:10.798394 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7868de-4e37-4191-a59e-b8ffcd5b68f6 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:10.811074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7868de-4e37-4191-a59e-b8ffcd5b68f6 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:10.818780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33962d1-ec9c-4d34-b98d-6892ef46a192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:10.819702 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33962d1-ec9c-4d34-b98d-6892ef46a192 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:10.819987 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33962d1-ec9c-4d34-b98d-6892ef46a192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:10.820263 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33962d1-ec9c-4d34-b98d-6892ef46a192 admin admin] PATH_INFO: `/v3/projects/a76209e0f3d248cba6b1b05adddbdf93/users/3a65885345544a8e8104d995ae20a552/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:10.841668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33962d1-ec9c-4d34-b98d-6892ef46a192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a76209e0f3d248cba6b1b05adddbdf93, user_id=3a65885345544a8e8104d995ae20a552, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:10.846862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33962d1-ec9c-4d34-b98d-6892ef46a192 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:10.851983 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:10.869113 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1022/2069] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:10 2022] PUT /identity/v3/projects/a76209e0f3d248cba6b1b05adddbdf93/users/3a65885345544a8e8104d995ae20a552/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:06:10.881381 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968dceb-07ef-4613-a95f-1dc0ae565aac None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:10.881704 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968dceb-07ef-4613-a95f-1dc0ae565aac None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:10.881938 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968dceb-07ef-4613-a95f-1dc0ae565aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:10.920466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d968dceb-07ef-4613-a95f-1dc0ae565aac None None] MFA Rules not processed for user `3a65885345544a8e8104d995ae20a552`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:10.923538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d968dceb-07ef-4613-a95f-1dc0ae565aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:10.967172 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:11.047016 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1048/2070] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:11.053319 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33962d1-ec9c-4d34-b98d-6892ef46a192 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:11.133431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33962d1-ec9c-4d34-b98d-6892ef46a192 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:11.144413 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a372ad-61f3-4d0f-8227-add0a796f495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:11.145394 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a372ad-61f3-4d0f-8227-add0a796f495 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:11.145825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a372ad-61f3-4d0f-8227-add0a796f495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:11.146286 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a372ad-61f3-4d0f-8227-add0a796f495 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:11.147180 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a372ad-61f3-4d0f-8227-add0a796f495 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:11.149220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a372ad-61f3-4d0f-8227-add0a796f495 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:11.200298 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1023/2071] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:11 2022] POST /identity/v3/projects => generated 381 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:06:11.207927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968dceb-07ef-4613-a95f-1dc0ae565aac None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:11.227211 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968dceb-07ef-4613-a95f-1dc0ae565aac None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:11.356004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20163b74-1a27-4f35-aa83-fcafabfc3b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:11.357120 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20163b74-1a27-4f35-aa83-fcafabfc3b89 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:11.357710 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20163b74-1a27-4f35-aa83-fcafabfc3b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:11.358244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20163b74-1a27-4f35-aa83-fcafabfc3b89 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:11.361132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20163b74-1a27-4f35-aa83-fcafabfc3b89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:11.365025 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20163b74-1a27-4f35-aa83-fcafabfc3b89 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:11.414302 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1049/2072] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:11 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:11.421715 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a372ad-61f3-4d0f-8227-add0a796f495 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:11.459251 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a372ad-61f3-4d0f-8227-add0a796f495 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:11.479124 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d1d30-9266-4571-ac48-108fc89d9628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:11.480346 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d1d30-9266-4571-ac48-108fc89d9628 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:11.480988 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d1d30-9266-4571-ac48-108fc89d9628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:11.481479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d1d30-9266-4571-ac48-108fc89d9628 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:11.483464 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d1d30-9266-4571-ac48-108fc89d9628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:11.489763 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d1d30-9266-4571-ac48-108fc89d9628 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:11.514157 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1024/2073] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:11.516773 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20163b74-1a27-4f35-aa83-fcafabfc3b89 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:11.552233 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20163b74-1a27-4f35-aa83-fcafabfc3b89 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:11.565136 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74aea9-9c43-4848-9913-059fe1e3fde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:11.568219 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74aea9-9c43-4848-9913-059fe1e3fde7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:11.568768 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74aea9-9c43-4848-9913-059fe1e3fde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:11.569257 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74aea9-9c43-4848-9913-059fe1e3fde7 admin admin] PATH_INFO: `/v3/projects/c9432e2d5d8a41cfb2b79b6ad5b12463/users/7d86f93f70394d52a6c6c8b85fe36448/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:11.602587 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74aea9-9c43-4848-9913-059fe1e3fde7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9432e2d5d8a41cfb2b79b6ad5b12463, user_id=7d86f93f70394d52a6c6c8b85fe36448, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:11.604453 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d1d30-9266-4571-ac48-108fc89d9628 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:11.607793 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-597d1d30-9266-4571-ac48-108fc89d9628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:11.611619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74aea9-9c43-4848-9913-059fe1e3fde7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:11.646829 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1050/2074] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:11 2022] PUT /identity/v3/projects/c9432e2d5d8a41cfb2b79b6ad5b12463/users/7d86f93f70394d52a6c6c8b85fe36448/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:11.652798 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d1d30-9266-4571-ac48-108fc89d9628 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:11.653104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74aea9-9c43-4848-9913-059fe1e3fde7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:11.753337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74aea9-9c43-4848-9913-059fe1e3fde7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:11.763903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13028d51-747c-4f0e-8fde-d349e858d604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:11.765543 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13028d51-747c-4f0e-8fde-d349e858d604 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:11.765953 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13028d51-747c-4f0e-8fde-d349e858d604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:11.766105 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13028d51-747c-4f0e-8fde-d349e858d604 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:11.766896 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13028d51-747c-4f0e-8fde-d349e858d604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:11.768604 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13028d51-747c-4f0e-8fde-d349e858d604 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:11.782104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1051/2075] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:11.787067 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13028d51-747c-4f0e-8fde-d349e858d604 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:11.811888 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13028d51-747c-4f0e-8fde-d349e858d604 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:11.814656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74cbd0-de32-485a-a463-edf6930a27b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:11.817121 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74cbd0-de32-485a-a463-edf6930a27b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:11.817749 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74cbd0-de32-485a-a463-edf6930a27b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:11.818367 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74cbd0-de32-485a-a463-edf6930a27b6 admin admin] PATH_INFO: `/v3/users/d87eaffe02374ca6b081f2d9e083af65` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:11.821140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74cbd0-de32-485a-a463-edf6930a27b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d87eaffe02374ca6b081f2d9e083af65)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:11.822743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22691e77-ac4c-4d3a-b441-0f383a38a2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:11.824309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22691e77-ac4c-4d3a-b441-0f383a38a2b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:11.824793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22691e77-ac4c-4d3a-b441-0f383a38a2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:11.825298 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22691e77-ac4c-4d3a-b441-0f383a38a2b5 admin admin] PATH_INFO: `/v3/projects/c9432e2d5d8a41cfb2b79b6ad5b12463/users/7d86f93f70394d52a6c6c8b85fe36448/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:11.826808 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74cbd0-de32-485a-a463-edf6930a27b6 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:11.829547 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22691e77-ac4c-4d3a-b441-0f383a38a2b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9432e2d5d8a41cfb2b79b6ad5b12463, user_id=7d86f93f70394d52a6c6c8b85fe36448, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:11.836632 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22691e77-ac4c-4d3a-b441-0f383a38a2b5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:11.864756 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1052/2076] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:11 2022] PUT /identity/v3/projects/c9432e2d5d8a41cfb2b79b6ad5b12463/users/7d86f93f70394d52a6c6c8b85fe36448/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:11.876548 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280190cc-09ab-4758-a694-fe9badd0099c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:11.877029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280190cc-09ab-4758-a694-fe9badd0099c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:11.878396 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280190cc-09ab-4758-a694-fe9badd0099c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:11.925429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:11.929338 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-8c74cbd0-de32-485a-a463-edf6930a27b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd87eaffe02374ca6b081f2d9e083af65'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:11.940907 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-8c74cbd0-de32-485a-a463-edf6930a27b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd87eaffe02374ca6b081f2d9e083af65'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:11.941750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-280190cc-09ab-4758-a694-fe9badd0099c None None] MFA Rules not processed for user `7d86f93f70394d52a6c6c8b85fe36448`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:11.941916 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-8c74cbd0-de32-485a-a463-edf6930a27b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd87eaffe02374ca6b081f2d9e083af65'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:11.943576 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-280190cc-09ab-4758-a694-fe9badd0099c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:11.945430 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-8c74cbd0-de32-485a-a463-edf6930a27b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd87eaffe02374ca6b081f2d9e083af65'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:11.958611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-8c74cbd0-de32-485a-a463-edf6930a27b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd87eaffe02374ca6b081f2d9e083af65'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:11.971007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:11.985549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1025/2077] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:11 2022] DELETE /identity/v3/users/d87eaffe02374ca6b081f2d9e083af65 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:11.991030 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd0d5d-e3bb-4e40-8586-fc2deb7ca23c None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:11.991673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd0d5d-e3bb-4e40-8586-fc2deb7ca23c None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:11.992351 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd0d5d-e3bb-4e40-8586-fc2deb7ca23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:12.019321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-45cd0d5d-e3bb-4e40-8586-fc2deb7ca23c None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:12.020568 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-45cd0d5d-e3bb-4e40-8586-fc2deb7ca23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:12.039038 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1053/2078] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:12.051998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280190cc-09ab-4758-a694-fe9badd0099c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:12.053451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-280190cc-09ab-4758-a694-fe9badd0099c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:12.117902 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1026/2079] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:06:12.122977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280190cc-09ab-4758-a694-fe9badd0099c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:12.132764 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd0d5d-e3bb-4e40-8586-fc2deb7ca23c None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:12.134438 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-45cd0d5d-e3bb-4e40-8586-fc2deb7ca23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:12.192572 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca17e3-b3c4-469a-b872-054cdd91dd0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:12.193148 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca17e3-b3c4-469a-b872-054cdd91dd0c service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:12.193443 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca17e3-b3c4-469a-b872-054cdd91dd0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:12.193738 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca17e3-b3c4-469a-b872-054cdd91dd0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:12.213426 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd0d5d-e3bb-4e40-8586-fc2deb7ca23c None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:12.219044 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84edd715-c1b4-4f8d-9c0f-d012b7456c44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:12.219933 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edd715-c1b4-4f8d-9c0f-d012b7456c44 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:12.220420 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edd715-c1b4-4f8d-9c0f-d012b7456c44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:12.220891 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edd715-c1b4-4f8d-9c0f-d012b7456c44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:12.231911 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84edd715-c1b4-4f8d-9c0f-d012b7456c44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:12.233453 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84edd715-c1b4-4f8d-9c0f-d012b7456c44 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:12.245053 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1027/2080] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:12.280368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca17e3-b3c4-469a-b872-054cdd91dd0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:12.282481 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca17e3-b3c4-469a-b872-054cdd91dd0c service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:12.293746 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1054/2081] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:06:12.621796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84edd715-c1b4-4f8d-9c0f-d012b7456c44 service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:12.623615 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-84edd715-c1b4-4f8d-9c0f-d012b7456c44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:12.645488 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84edd715-c1b4-4f8d-9c0f-d012b7456c44 service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:12.652005 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2674c6-2149-42c3-8089-8524f2ae8e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:12.654585 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2674c6-2149-42c3-8089-8524f2ae8e9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:12.655171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2674c6-2149-42c3-8089-8524f2ae8e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:12.655809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2674c6-2149-42c3-8089-8524f2ae8e9f admin admin] PATH_INFO: `/v3/projects/db354d763e214ca4a62b0f964ee5b60f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:12.658144 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2674c6-2149-42c3-8089-8524f2ae8e9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=db354d763e214ca4a62b0f964ee5b60f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:12.660477 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2674c6-2149-42c3-8089-8524f2ae8e9f admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:12.732290 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-bf2674c6-2149-42c3-8089-8524f2ae8e9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db354d763e214ca4a62b0f964ee5b60f', 'request_id': 'req-bf2674c6-2149-42c3-8089-8524f2ae8e9f'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:12.743012 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-bf2674c6-2149-42c3-8089-8524f2ae8e9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db354d763e214ca4a62b0f964ee5b60f', 'request_id': 'req-bf2674c6-2149-42c3-8089-8524f2ae8e9f'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:12.750348 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-bf2674c6-2149-42c3-8089-8524f2ae8e9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db354d763e214ca4a62b0f964ee5b60f', 'request_id': 'req-bf2674c6-2149-42c3-8089-8524f2ae8e9f'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:12.763082 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-bf2674c6-2149-42c3-8089-8524f2ae8e9f admin admin] The token cache is being invalidate because project db354d763e214ca4a62b0f964ee5b60f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:12.763379 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-bf2674c6-2149-42c3-8089-8524f2ae8e9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:12.764064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-bf2674c6-2149-42c3-8089-8524f2ae8e9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:12.766482 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1028/2082] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:12 2022] DELETE /identity/v3/projects/db354d763e214ca4a62b0f964ee5b60f => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:12.775482 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca17e3-b3c4-469a-b872-054cdd91dd0c service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:12.787068 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:12.799783 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca17e3-b3c4-469a-b872-054cdd91dd0c service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:12.816357 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2674c6-2149-42c3-8089-8524f2ae8e9f admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:12.818082 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-bf2674c6-2149-42c3-8089-8524f2ae8e9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:12.839085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:12.901644 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e763d11-06c5-47e9-ab35-68e9140c48e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:12.903957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e763d11-06c5-47e9-ab35-68e9140c48e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:12.904645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e763d11-06c5-47e9-ab35-68e9140c48e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:12.904820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2674c6-2149-42c3-8089-8524f2ae8e9f admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:12.907260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e763d11-06c5-47e9-ab35-68e9140c48e4 admin admin] PATH_INFO: `/v3/users/e5585e732f93478e87be9f8f8815efe4` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:12.912473 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e763d11-06c5-47e9-ab35-68e9140c48e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5585e732f93478e87be9f8f8815efe4)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:12.914532 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e763d11-06c5-47e9-ab35-68e9140c48e4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:12.919802 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ede7a3-9ef5-42be-9760-3d971a0965b1 tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d86f93f70394d52a6c6c8b85fe36448', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9432e2d5d8a41cfb2b79b6ad5b12463', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:12.922028 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ede7a3-9ef5-42be-9760-3d971a0965b1 tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:12.922747 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ede7a3-9ef5-42be-9760-3d971a0965b1 tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:12.923401 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ede7a3-9ef5-42be-9760-3d971a0965b1 tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:12.925591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ede7a3-9ef5-42be-9760-3d971a0965b1 tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:06:12.925591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ede7a3-9ef5-42be-9760-3d971a0965b1 tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:12.928970 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ede7a3-9ef5-42be-9760-3d971a0965b1 tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:12.954325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1029/2083] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:06:12.958269 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ede7a3-9ef5-42be-9760-3d971a0965b1 tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:12.973707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ede7a3-9ef5-42be-9760-3d971a0965b1 tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:12.981549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f017db-3eca-4ce6-896e-4d5790733150 tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d86f93f70394d52a6c6c8b85fe36448', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9432e2d5d8a41cfb2b79b6ad5b12463', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:12.982834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f017db-3eca-4ce6-896e-4d5790733150 tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:12.983300 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f017db-3eca-4ce6-896e-4d5790733150 tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:12.985467 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f017db-3eca-4ce6-896e-4d5790733150 tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:12.986895 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f017db-3eca-4ce6-896e-4d5790733150 tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:12.991033 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f017db-3eca-4ce6-896e-4d5790733150 tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:12.996193 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8e763d11-06c5-47e9-ab35-68e9140c48e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5585e732f93478e87be9f8f8815efe4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:13.012916 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8e763d11-06c5-47e9-ab35-68e9140c48e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5585e732f93478e87be9f8f8815efe4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:13.033165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8e763d11-06c5-47e9-ab35-68e9140c48e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5585e732f93478e87be9f8f8815efe4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:13.034053 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8e763d11-06c5-47e9-ab35-68e9140c48e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5585e732f93478e87be9f8f8815efe4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:13.035155 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8e763d11-06c5-47e9-ab35-68e9140c48e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5585e732f93478e87be9f8f8815efe4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:13.038251 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1030/2084] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:12 2022] POST /identity/v3/projects => generated 355 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:06:13.043198 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f017db-3eca-4ce6-896e-4d5790733150 tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:13.044706 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-a0f017db-3eca-4ce6-896e-4d5790733150 tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:13.051712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1055/2085] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:12 2022] DELETE /identity/v3/users/e5585e732f93478e87be9f8f8815efe4 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:13.132005 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f017db-3eca-4ce6-896e-4d5790733150 tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:13.212974 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74d1bc-a6bc-481d-b555-576b177ccf5b tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d86f93f70394d52a6c6c8b85fe36448', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9432e2d5d8a41cfb2b79b6ad5b12463', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:13.213673 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74d1bc-a6bc-481d-b555-576b177ccf5b tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:13.213966 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74d1bc-a6bc-481d-b555-576b177ccf5b tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:13.214431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74d1bc-a6bc-481d-b555-576b177ccf5b tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:13.215175 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74d1bc-a6bc-481d-b555-576b177ccf5b tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:06:13.215496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74d1bc-a6bc-481d-b555-576b177ccf5b tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:13.217153 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74d1bc-a6bc-481d-b555-576b177ccf5b tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:13.238265 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1031/2086] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:06:13.289920 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e763d11-06c5-47e9-ab35-68e9140c48e4 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:13.301806 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e763d11-06c5-47e9-ab35-68e9140c48e4 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:13.307168 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95666a-b214-4764-b996-ff4cdb09050a tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d86f93f70394d52a6c6c8b85fe36448', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9432e2d5d8a41cfb2b79b6ad5b12463', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:13.308072 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95666a-b214-4764-b996-ff4cdb09050a tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:13.308365 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95666a-b214-4764-b996-ff4cdb09050a tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:13.308640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95666a-b214-4764-b996-ff4cdb09050a tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] PATH_INFO: `/v3/projects/d4607db82d11469b9774d7aef286e692` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:13.309914 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95666a-b214-4764-b996-ff4cdb09050a tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d4607db82d11469b9774d7aef286e692)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:13.312041 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95666a-b214-4764-b996-ff4cdb09050a tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:13.369104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-db95666a-b214-4764-b996-ff4cdb09050a tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4607db82d11469b9774d7aef286e692', 'request_id': 'req-db95666a-b214-4764-b996-ff4cdb09050a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:13.378570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-db95666a-b214-4764-b996-ff4cdb09050a tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4607db82d11469b9774d7aef286e692', 'request_id': 'req-db95666a-b214-4764-b996-ff4cdb09050a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:13.387651 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-db95666a-b214-4764-b996-ff4cdb09050a tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4607db82d11469b9774d7aef286e692', 'request_id': 'req-db95666a-b214-4764-b996-ff4cdb09050a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:13.396838 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74d1bc-a6bc-481d-b555-576b177ccf5b tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:13.399862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-cb74d1bc-a6bc-481d-b555-576b177ccf5b tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:13.407396 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-db95666a-b214-4764-b996-ff4cdb09050a tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] The token cache is being invalidate because project d4607db82d11469b9774d7aef286e692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:13.407681 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-db95666a-b214-4764-b996-ff4cdb09050a tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:13.408394 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-db95666a-b214-4764-b996-ff4cdb09050a tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:13.410220 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1056/2087] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:13 2022] DELETE /identity/v3/projects/d4607db82d11469b9774d7aef286e692 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:13.457219 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:13.512028 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74d1bc-a6bc-481d-b555-576b177ccf5b tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:13.611263 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f7551-7548-461d-abe5-71b9d3552ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:13.612620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f7551-7548-461d-abe5-71b9d3552ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:13.613349 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f7551-7548-461d-abe5-71b9d3552ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:13.613637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f7551-7548-461d-abe5-71b9d3552ea4 admin admin] PATH_INFO: `/v3/projects/8322d85b7ebd468ca4ee85d9d94e19be` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:13.615153 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f7551-7548-461d-abe5-71b9d3552ea4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8322d85b7ebd468ca4ee85d9d94e19be)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:13.617103 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f7551-7548-461d-abe5-71b9d3552ea4 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:13.685012 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-938f7551-7548-461d-abe5-71b9d3552ea4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8322d85b7ebd468ca4ee85d9d94e19be', 'request_id': 'req-938f7551-7548-461d-abe5-71b9d3552ea4'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:13.695206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-938f7551-7548-461d-abe5-71b9d3552ea4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8322d85b7ebd468ca4ee85d9d94e19be', 'request_id': 'req-938f7551-7548-461d-abe5-71b9d3552ea4'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:13.703410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-938f7551-7548-461d-abe5-71b9d3552ea4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8322d85b7ebd468ca4ee85d9d94e19be', 'request_id': 'req-938f7551-7548-461d-abe5-71b9d3552ea4'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:13.706489 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95666a-b214-4764-b996-ff4cdb09050a tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:13.707902 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-db95666a-b214-4764-b996-ff4cdb09050a tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:13.718433 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:13.723682 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-938f7551-7548-461d-abe5-71b9d3552ea4 admin admin] The token cache is being invalidate because project 8322d85b7ebd468ca4ee85d9d94e19be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:13.723826 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95666a-b214-4764-b996-ff4cdb09050a tempest-VolumeQuotaClassesTest-1386824177 tempest-VolumeQuotaClassesTest-1386824177-project-admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:13.724180 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-938f7551-7548-461d-abe5-71b9d3552ea4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:13.725085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-938f7551-7548-461d-abe5-71b9d3552ea4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:13.727255 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1032/2088] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:13 2022] DELETE /identity/v3/projects/8322d85b7ebd468ca4ee85d9d94e19be => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:13.732317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd7ae6-a62e-44f1-a992-9dc2d0bc4f0e None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:13.732611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd7ae6-a62e-44f1-a992-9dc2d0bc4f0e None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:13.732901 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd7ae6-a62e-44f1-a992-9dc2d0bc4f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:13.753275 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-78bd7ae6-a62e-44f1-a992-9dc2d0bc4f0e None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:13.754556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-78bd7ae6-a62e-44f1-a992-9dc2d0bc4f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:13.791089 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb12861-4fd8-4299-aaac-1b4cd88d9f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:13.791752 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb12861-4fd8-4299-aaac-1b4cd88d9f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:13.792041 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb12861-4fd8-4299-aaac-1b4cd88d9f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:13.792313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb12861-4fd8-4299-aaac-1b4cd88d9f3b admin admin] PATH_INFO: `/v3/users/3a65885345544a8e8104d995ae20a552` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:13.793528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb12861-4fd8-4299-aaac-1b4cd88d9f3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a65885345544a8e8104d995ae20a552)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:13.795502 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb12861-4fd8-4299-aaac-1b4cd88d9f3b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:13.825613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1033/2089] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:06:13.834596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac25f81-181f-4c80-acd1-a4c35c0ffeed None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:13.835165 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac25f81-181f-4c80-acd1-a4c35c0ffeed None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:13.835778 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac25f81-181f-4c80-acd1-a4c35c0ffeed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:13.854432 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:13.863018 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-aac25f81-181f-4c80-acd1-a4c35c0ffeed None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:13.863656 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8bb12861-4fd8-4299-aaac-1b4cd88d9f3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a65885345544a8e8104d995ae20a552'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:13.865059 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-aac25f81-181f-4c80-acd1-a4c35c0ffeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:13.871018 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8bb12861-4fd8-4299-aaac-1b4cd88d9f3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a65885345544a8e8104d995ae20a552'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:13.877635 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1034/2090] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:13.879287 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac25f81-181f-4c80-acd1-a4c35c0ffeed None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:13.884569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8bb12861-4fd8-4299-aaac-1b4cd88d9f3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a65885345544a8e8104d995ae20a552'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:13.885297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8bb12861-4fd8-4299-aaac-1b4cd88d9f3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a65885345544a8e8104d995ae20a552'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:13.885913 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8bb12861-4fd8-4299-aaac-1b4cd88d9f3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a65885345544a8e8104d995ae20a552'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:13.890616 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac25f81-181f-4c80-acd1-a4c35c0ffeed None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:13.896546 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0f7de-1935-48e1-9e7e-4df425881cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:13.897225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0f7de-1935-48e1-9e7e-4df425881cfb admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:13.897509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0f7de-1935-48e1-9e7e-4df425881cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:13.897791 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0f7de-1935-48e1-9e7e-4df425881cfb admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:13.898563 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0f7de-1935-48e1-9e7e-4df425881cfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:06:13.898874 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0f7de-1935-48e1-9e7e-4df425881cfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:13.900525 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0f7de-1935-48e1-9e7e-4df425881cfb admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:13.901026 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1057/2091] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:13 2022] DELETE /identity/v3/users/3a65885345544a8e8104d995ae20a552 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:06:13.906487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27fc36-71a4-40db-8267-252704afc7e8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:13.906779 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27fc36-71a4-40db-8267-252704afc7e8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:13.907053 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27fc36-71a4-40db-8267-252704afc7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:13.915501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1035/2092] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:06:13.920429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0f7de-1935-48e1-9e7e-4df425881cfb admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:13.922356 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-5fa0f7de-1935-48e1-9e7e-4df425881cfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:13.925452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6e27fc36-71a4-40db-8267-252704afc7e8 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:13.926661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6e27fc36-71a4-40db-8267-252704afc7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:13.935623 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0f7de-1935-48e1-9e7e-4df425881cfb admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:13.992216 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1058/2093] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:06:13.999625 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba3eff-9f0f-47fa-8df9-c4816f8e23cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:14.000418 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba3eff-9f0f-47fa-8df9-c4816f8e23cc admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:14.000701 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba3eff-9f0f-47fa-8df9-c4816f8e23cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:14.000981 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba3eff-9f0f-47fa-8df9-c4816f8e23cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:14.001765 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba3eff-9f0f-47fa-8df9-c4816f8e23cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:14.002842 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27fc36-71a4-40db-8267-252704afc7e8 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:14.003753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba3eff-9f0f-47fa-8df9-c4816f8e23cc admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:14.004216 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6e27fc36-71a4-40db-8267-252704afc7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:14.032530 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1036/2094] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:13 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:06:14.036797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba3eff-9f0f-47fa-8df9-c4816f8e23cc admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:14.039787 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:14.080278 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27fc36-71a4-40db-8267-252704afc7e8 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:14.107208 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba3eff-9f0f-47fa-8df9-c4816f8e23cc admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:14.112069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01eb39f-d2da-4638-b3b9-a8bb6252e73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:14.112745 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01eb39f-d2da-4638-b3b9-a8bb6252e73a admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:14.113035 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01eb39f-d2da-4638-b3b9-a8bb6252e73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:14.113316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01eb39f-d2da-4638-b3b9-a8bb6252e73a admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:14.114025 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01eb39f-d2da-4638-b3b9-a8bb6252e73a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:14.116004 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01eb39f-d2da-4638-b3b9-a8bb6252e73a admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:14.137686 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4af15d-ecda-43ab-b954-3404aab70f9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:14.138826 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4af15d-ecda-43ab-b954-3404aab70f9b service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:14.139025 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4af15d-ecda-43ab-b954-3404aab70f9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:14.140334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4af15d-ecda-43ab-b954-3404aab70f9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:14.141851 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1037/2095] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:14 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:14.146023 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01eb39f-d2da-4638-b3b9-a8bb6252e73a admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:14.151909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4af15d-ecda-43ab-b954-3404aab70f9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:14.153605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4af15d-ecda-43ab-b954-3404aab70f9b service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:14.158516 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01eb39f-d2da-4638-b3b9-a8bb6252e73a admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:14.164460 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e7ee1-c87c-46b1-b181-c57a1195f21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:14.165185 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e7ee1-c87c-46b1-b181-c57a1195f21f admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:14.165481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e7ee1-c87c-46b1-b181-c57a1195f21f admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:14.165763 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e7ee1-c87c-46b1-b181-c57a1195f21f admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:14.166543 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e7ee1-c87c-46b1-b181-c57a1195f21f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:14.167228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1059/2096] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:06:14.168177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e7ee1-c87c-46b1-b181-c57a1195f21f admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:14.179226 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1038/2097] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:14.183480 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4af15d-ecda-43ab-b954-3404aab70f9b service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:14.195056 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4af15d-ecda-43ab-b954-3404aab70f9b service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:14.202789 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1768a-45a0-40db-8650-59e6b11a5609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:14.202789 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1768a-45a0-40db-8650-59e6b11a5609 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:14.202789 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1768a-45a0-40db-8650-59e6b11a5609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:14.202789 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1768a-45a0-40db-8650-59e6b11a5609 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:14.203548 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1768a-45a0-40db-8650-59e6b11a5609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:14.205289 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1768a-45a0-40db-8650-59e6b11a5609 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:14.218492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1060/2098] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:06:14.222885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e7ee1-c87c-46b1-b181-c57a1195f21f admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:14.237581 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e7ee1-c87c-46b1-b181-c57a1195f21f admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:14.244031 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3bfd20-7d23-4047-a686-aef4b9fca731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:14.244967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3bfd20-7d23-4047-a686-aef4b9fca731 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:14.245295 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3bfd20-7d23-4047-a686-aef4b9fca731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:14.245614 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3bfd20-7d23-4047-a686-aef4b9fca731 admin admin] PATH_INFO: `/v3/projects/f1056091fba74e56a14f77d588f33afc/users/d39a402d0e7b4220b787d9c8ff30a1cb/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:14.268547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3bfd20-7d23-4047-a686-aef4b9fca731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1056091fba74e56a14f77d588f33afc, user_id=d39a402d0e7b4220b787d9c8ff30a1cb, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:14.272543 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3bfd20-7d23-4047-a686-aef4b9fca731 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:14.294243 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1039/2099] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:14 2022] PUT /identity/v3/projects/f1056091fba74e56a14f77d588f33afc/users/d39a402d0e7b4220b787d9c8ff30a1cb/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:06:14.307277 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c7733-ec62-4c0a-bd08-0b5881e3aae1 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:14.307660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c7733-ec62-4c0a-bd08-0b5881e3aae1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:14.308101 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c7733-ec62-4c0a-bd08-0b5881e3aae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:14.361082 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-bf0c7733-ec62-4c0a-bd08-0b5881e3aae1 None None] MFA Rules not processed for user `d39a402d0e7b4220b787d9c8ff30a1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:14.362416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bf0c7733-ec62-4c0a-bd08-0b5881e3aae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:14.437488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1061/2100] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:06:14.442297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3bfd20-7d23-4047-a686-aef4b9fca731 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:14.475361 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:14.508138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c7733-ec62-4c0a-bd08-0b5881e3aae1 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:14.511060 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bf0c7733-ec62-4c0a-bd08-0b5881e3aae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:14.535018 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3bfd20-7d23-4047-a686-aef4b9fca731 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:14.541501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7e6ee-0b4f-416e-b58d-6ce171149fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:14.542372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7e6ee-0b4f-416e-b58d-6ce171149fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:14.542664 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7e6ee-0b4f-416e-b58d-6ce171149fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:14.542945 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7e6ee-0b4f-416e-b58d-6ce171149fd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:14.543668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf7e6ee-0b4f-416e-b58d-6ce171149fd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:14.545602 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf7e6ee-0b4f-416e-b58d-6ce171149fd9 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:14.588541 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1040/2101] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:14 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:14.595387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7e6ee-0b4f-416e-b58d-6ce171149fd9 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:14.629035 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c7733-ec62-4c0a-bd08-0b5881e3aae1 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:14.641082 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:14.737728 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7e6ee-0b4f-416e-b58d-6ce171149fd9 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:14.749326 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61ce6b-ea36-4b5b-9ad1-4d065553e4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:14.750402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61ce6b-ea36-4b5b-9ad1-4d065553e4a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:14.750699 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61ce6b-ea36-4b5b-9ad1-4d065553e4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:14.750985 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61ce6b-ea36-4b5b-9ad1-4d065553e4a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:14.751709 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61ce6b-ea36-4b5b-9ad1-4d065553e4a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:14.754361 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61ce6b-ea36-4b5b-9ad1-4d065553e4a3 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:14.760679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b74a8-8c7b-42b9-a726-05527636fd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:14.761798 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b74a8-8c7b-42b9-a726-05527636fd43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:14.762238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b74a8-8c7b-42b9-a726-05527636fd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:14.763902 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b74a8-8c7b-42b9-a726-05527636fd43 admin admin] PATH_INFO: `/v3/projects/a76209e0f3d248cba6b1b05adddbdf93` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:14.766609 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b74a8-8c7b-42b9-a726-05527636fd43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a76209e0f3d248cba6b1b05adddbdf93)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:14.769606 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b74a8-8c7b-42b9-a726-05527636fd43 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:14.802427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1041/2102] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:14 2022] POST /identity/v3/users => generated 294 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:14.808364 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61ce6b-ea36-4b5b-9ad1-4d065553e4a3 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:14.828314 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61ce6b-ea36-4b5b-9ad1-4d065553e4a3 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:14.834584 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b075f-530d-4b6c-b857-13355b8c5df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:14.835435 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b075f-530d-4b6c-b857-13355b8c5df6 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:14.835830 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b075f-530d-4b6c-b857-13355b8c5df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:14.836426 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b075f-530d-4b6c-b857-13355b8c5df6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:14.837599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b075f-530d-4b6c-b857-13355b8c5df6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:14.839523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b075f-530d-4b6c-b857-13355b8c5df6 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:14.883593 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1042/2103] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:14.888307 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-912b74a8-8c7b-42b9-a726-05527636fd43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a76209e0f3d248cba6b1b05adddbdf93', 'request_id': 'req-912b74a8-8c7b-42b9-a726-05527636fd43'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:14.888905 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b075f-530d-4b6c-b857-13355b8c5df6 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:14.900526 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-912b74a8-8c7b-42b9-a726-05527636fd43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a76209e0f3d248cba6b1b05adddbdf93', 'request_id': 'req-912b74a8-8c7b-42b9-a726-05527636fd43'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:14.901011 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b075f-530d-4b6c-b857-13355b8c5df6 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:14.910078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-912b74a8-8c7b-42b9-a726-05527636fd43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a76209e0f3d248cba6b1b05adddbdf93', 'request_id': 'req-912b74a8-8c7b-42b9-a726-05527636fd43'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:14.932446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-912b74a8-8c7b-42b9-a726-05527636fd43 admin admin] The token cache is being invalidate because project a76209e0f3d248cba6b1b05adddbdf93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:14.932738 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-912b74a8-8c7b-42b9-a726-05527636fd43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:14.933472 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-912b74a8-8c7b-42b9-a726-05527636fd43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:14.938651 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1062/2104] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:14 2022] DELETE /identity/v3/projects/a76209e0f3d248cba6b1b05adddbdf93 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:14.940784 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b74a8-8c7b-42b9-a726-05527636fd43 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:14.943797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-912b74a8-8c7b-42b9-a726-05527636fd43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:15.047669 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa348dfe-cbc6-41b3-ab5e-d5d38ed2de8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:15.048608 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa348dfe-cbc6-41b3-ab5e-d5d38ed2de8d admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:15.049212 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa348dfe-cbc6-41b3-ab5e-d5d38ed2de8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:15.049212 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa348dfe-cbc6-41b3-ab5e-d5d38ed2de8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:15.049927 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa348dfe-cbc6-41b3-ab5e-d5d38ed2de8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:15.051874 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa348dfe-cbc6-41b3-ab5e-d5d38ed2de8d admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:15.066276 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1043/2105] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:15.069737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa348dfe-cbc6-41b3-ab5e-d5d38ed2de8d admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:15.073603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-fa348dfe-cbc6-41b3-ab5e-d5d38ed2de8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:15.076787 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b74a8-8c7b-42b9-a726-05527636fd43 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:15.083895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8c6ca-0b09-4b2b-905e-dcd476ce450c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:15.084959 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8c6ca-0b09-4b2b-905e-dcd476ce450c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:15.085398 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8c6ca-0b09-4b2b-905e-dcd476ce450c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:15.085863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8c6ca-0b09-4b2b-905e-dcd476ce450c admin admin] PATH_INFO: `/v3/users/7d86f93f70394d52a6c6c8b85fe36448` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:15.088357 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8c6ca-0b09-4b2b-905e-dcd476ce450c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d86f93f70394d52a6c6c8b85fe36448)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:15.091812 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8c6ca-0b09-4b2b-905e-dcd476ce450c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:15.101823 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa348dfe-cbc6-41b3-ab5e-d5d38ed2de8d admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:15.112145 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce904a3-f608-47ce-8eaa-d124be955fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:15.113519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce904a3-f608-47ce-8eaa-d124be955fc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:15.114172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce904a3-f608-47ce-8eaa-d124be955fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:15.114795 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce904a3-f608-47ce-8eaa-d124be955fc0 admin admin] PATH_INFO: `/v3/projects/5bf803da3c0343ed831c6ded0df4fba8/users/ff6b4dd95538450e8ff180122a0be531/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:15.137482 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce904a3-f608-47ce-8eaa-d124be955fc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bf803da3c0343ed831c6ded0df4fba8, user_id=ff6b4dd95538450e8ff180122a0be531, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:15.141682 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce904a3-f608-47ce-8eaa-d124be955fc0 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:15.160265 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1044/2106] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:15 2022] PUT /identity/v3/projects/5bf803da3c0343ed831c6ded0df4fba8/users/ff6b4dd95538450e8ff180122a0be531/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:06:15.179048 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92009225-e7e9-4de7-80d0-0d258cd5d0ad None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:15.179712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92009225-e7e9-4de7-80d0-0d258cd5d0ad None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:15.180130 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92009225-e7e9-4de7-80d0-0d258cd5d0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:15.180778 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:15.188590 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:15.199568 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-10d8c6ca-0b09-4b2b-905e-dcd476ce450c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d86f93f70394d52a6c6c8b85fe36448'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:15.212233 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-10d8c6ca-0b09-4b2b-905e-dcd476ce450c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d86f93f70394d52a6c6c8b85fe36448'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:15.224172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-92009225-e7e9-4de7-80d0-0d258cd5d0ad None None] MFA Rules not processed for user `ff6b4dd95538450e8ff180122a0be531`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:15.225576 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-92009225-e7e9-4de7-80d0-0d258cd5d0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:15.240064 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-10d8c6ca-0b09-4b2b-905e-dcd476ce450c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d86f93f70394d52a6c6c8b85fe36448'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:15.241104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-10d8c6ca-0b09-4b2b-905e-dcd476ce450c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d86f93f70394d52a6c6c8b85fe36448'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:15.241950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-10d8c6ca-0b09-4b2b-905e-dcd476ce450c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d86f93f70394d52a6c6c8b85fe36448'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:15.261329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1063/2107] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:14 2022] DELETE /identity/v3/users/7d86f93f70394d52a6c6c8b85fe36448 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:15.339008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1045/2108] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:06:15.346886 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8c6ca-0b09-4b2b-905e-dcd476ce450c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:15.349142 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-10d8c6ca-0b09-4b2b-905e-dcd476ce450c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:15.507208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8c6ca-0b09-4b2b-905e-dcd476ce450c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:15.526468 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9e5e8-14be-4c8d-9cde-56dbcd6c4231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:15.532756 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9e5e8-14be-4c8d-9cde-56dbcd6c4231 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:15.533256 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9e5e8-14be-4c8d-9cde-56dbcd6c4231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:15.533553 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9e5e8-14be-4c8d-9cde-56dbcd6c4231 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:15.536114 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e9e5e8-14be-4c8d-9cde-56dbcd6c4231 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:15.538482 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e9e5e8-14be-4c8d-9cde-56dbcd6c4231 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:15.591629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1064/2109] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:15 2022] POST /identity/v3/projects => generated 387 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:15.597349 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92009225-e7e9-4de7-80d0-0d258cd5d0ad None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:15.659620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9e5e8-14be-4c8d-9cde-56dbcd6c4231 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:15.661616 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-49e9e5e8-14be-4c8d-9cde-56dbcd6c4231 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:15.696957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92009225-e7e9-4de7-80d0-0d258cd5d0ad None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:15.737203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9e5e8-14be-4c8d-9cde-56dbcd6c4231 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:15.742478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dfdfe-ee82-4353-ba6b-57c69743358b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:15.743354 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dfdfe-ee82-4353-ba6b-57c69743358b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:15.743852 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dfdfe-ee82-4353-ba6b-57c69743358b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:15.744399 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dfdfe-ee82-4353-ba6b-57c69743358b admin admin] PATH_INFO: `/v3/projects/c9432e2d5d8a41cfb2b79b6ad5b12463` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:15.745380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2dfdfe-ee82-4353-ba6b-57c69743358b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9432e2d5d8a41cfb2b79b6ad5b12463)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:15.747345 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2dfdfe-ee82-4353-ba6b-57c69743358b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:15.757248 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a369540-eda4-4e12-9e6a-57f318869ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:15.758084 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a369540-eda4-4e12-9e6a-57f318869ced admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:15.758409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a369540-eda4-4e12-9e6a-57f318869ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:15.758689 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a369540-eda4-4e12-9e6a-57f318869ced admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:15.759389 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a369540-eda4-4e12-9e6a-57f318869ced admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:15.761475 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a369540-eda4-4e12-9e6a-57f318869ced admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:15.782205 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1046/2110] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:15 2022] POST /identity/v3/users => generated 300 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:15.786360 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a369540-eda4-4e12-9e6a-57f318869ced admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:15.796803 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a369540-eda4-4e12-9e6a-57f318869ced admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:15.797566 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:15.800905 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fa2dfdfe-ee82-4353-ba6b-57c69743358b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9432e2d5d8a41cfb2b79b6ad5b12463', 'request_id': 'req-fa2dfdfe-ee82-4353-ba6b-57c69743358b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:15.807562 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fa2dfdfe-ee82-4353-ba6b-57c69743358b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9432e2d5d8a41cfb2b79b6ad5b12463', 'request_id': 'req-fa2dfdfe-ee82-4353-ba6b-57c69743358b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:15.816002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fa2dfdfe-ee82-4353-ba6b-57c69743358b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9432e2d5d8a41cfb2b79b6ad5b12463', 'request_id': 'req-fa2dfdfe-ee82-4353-ba6b-57c69743358b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:15.826985 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:15.831175 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fa2dfdfe-ee82-4353-ba6b-57c69743358b admin admin] The token cache is being invalidate because project c9432e2d5d8a41cfb2b79b6ad5b12463 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:15.831479 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fa2dfdfe-ee82-4353-ba6b-57c69743358b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:15.832163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fa2dfdfe-ee82-4353-ba6b-57c69743358b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:15.833762 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1065/2111] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:15 2022] DELETE /identity/v3/projects/c9432e2d5d8a41cfb2b79b6ad5b12463 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:15.839840 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802ae60-dc21-458c-8674-ce5a564bf141 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:15.840130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802ae60-dc21-458c-8674-ce5a564bf141 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:15.840449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802ae60-dc21-458c-8674-ce5a564bf141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:15.859930 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-9802ae60-dc21-458c-8674-ce5a564bf141 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:15.861164 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9802ae60-dc21-458c-8674-ce5a564bf141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:15.861608 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cce40-bda4-4c0f-b521-b2f18bd06650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:15.862350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cce40-bda4-4c0f-b521-b2f18bd06650 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:15.862725 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cce40-bda4-4c0f-b521-b2f18bd06650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:15.862954 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cce40-bda4-4c0f-b521-b2f18bd06650 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:15.863689 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833cce40-bda4-4c0f-b521-b2f18bd06650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:15.865296 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833cce40-bda4-4c0f-b521-b2f18bd06650 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:15.872707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1066/2112] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:15.877396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1047/2113] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:06:15.882071 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cce40-bda4-4c0f-b521-b2f18bd06650 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:15.882280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664b960-e7ba-48e6-ba87-d3acc36d37c7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:15.882655 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664b960-e7ba-48e6-ba87-d3acc36d37c7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:15.882990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664b960-e7ba-48e6-ba87-d3acc36d37c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:15.884331 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-833cce40-bda4-4c0f-b521-b2f18bd06650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:15.900704 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cce40-bda4-4c0f-b521-b2f18bd06650 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:15.906522 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e0c72-1ce3-4cd2-ba14-ffd397a22601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:15.907178 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1664b960-e7ba-48e6-ba87-d3acc36d37c7 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:15.907658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e0c72-1ce3-4cd2-ba14-ffd397a22601 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:15.907747 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e0c72-1ce3-4cd2-ba14-ffd397a22601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:15.908051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e0c72-1ce3-4cd2-ba14-ffd397a22601 admin admin] PATH_INFO: `/v3/projects/90c0d211518e4254acebf59e345921ea/users/f9781166ed354aa2b36dfa2e90c53617/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:15.908442 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1664b960-e7ba-48e6-ba87-d3acc36d37c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:15.917402 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1067/2114] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:15.921436 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664b960-e7ba-48e6-ba87-d3acc36d37c7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:15.924776 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28e0c72-1ce3-4cd2-ba14-ffd397a22601 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90c0d211518e4254acebf59e345921ea, user_id=f9781166ed354aa2b36dfa2e90c53617, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:15.927503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28e0c72-1ce3-4cd2-ba14-ffd397a22601 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:15.931740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664b960-e7ba-48e6-ba87-d3acc36d37c7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:15.939043 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9afdac-c00a-45fd-bb7e-e1ce3983e44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:15.939881 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9afdac-c00a-45fd-bb7e-e1ce3983e44d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:15.940181 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9afdac-c00a-45fd-bb7e-e1ce3983e44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:15.940623 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9afdac-c00a-45fd-bb7e-e1ce3983e44d admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:15.941546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9afdac-c00a-45fd-bb7e-e1ce3983e44d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:06:15.941999 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9afdac-c00a-45fd-bb7e-e1ce3983e44d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:15.944000 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1048/2115] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:15 2022] PUT /identity/v3/projects/90c0d211518e4254acebf59e345921ea/users/f9781166ed354aa2b36dfa2e90c53617/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:06:15.944612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9afdac-c00a-45fd-bb7e-e1ce3983e44d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:15.947843 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e0c72-1ce3-4cd2-ba14-ffd397a22601 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:15.963950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1068/2116] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:06:15.965203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9afdac-c00a-45fd-bb7e-e1ce3983e44d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:15.977439 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9afdac-c00a-45fd-bb7e-e1ce3983e44d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:16.035390 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e0c72-1ce3-4cd2-ba14-ffd397a22601 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:16.041942 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617098a-b774-46a7-8769-ebc70d5583b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:16.042663 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617098a-b774-46a7-8769-ebc70d5583b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:16.042951 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617098a-b774-46a7-8769-ebc70d5583b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:16.043245 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617098a-b774-46a7-8769-ebc70d5583b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:16.043977 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617098a-b774-46a7-8769-ebc70d5583b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:16.045584 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617098a-b774-46a7-8769-ebc70d5583b0 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:16.054751 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c8a52-5204-4c7d-b6c1-a26b0d119a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:16.055418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c8a52-5204-4c7d-b6c1-a26b0d119a93 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:16.055705 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c8a52-5204-4c7d-b6c1-a26b0d119a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:16.055984 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c8a52-5204-4c7d-b6c1-a26b0d119a93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:16.056647 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1049/2117] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:06:16.058277 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c8a52-5204-4c7d-b6c1-a26b0d119a93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:16.060376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c8a52-5204-4c7d-b6c1-a26b0d119a93 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:16.060873 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617098a-b774-46a7-8769-ebc70d5583b0 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:16.071604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617098a-b774-46a7-8769-ebc70d5583b0 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:16.077582 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520d68e-12ab-4069-a36f-c777cf690d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:16.078698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520d68e-12ab-4069-a36f-c777cf690d6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:16.079004 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520d68e-12ab-4069-a36f-c777cf690d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:16.079283 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520d68e-12ab-4069-a36f-c777cf690d6d admin admin] PATH_INFO: `/v3/projects/90c0d211518e4254acebf59e345921ea/users/f9781166ed354aa2b36dfa2e90c53617/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:16.081757 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5520d68e-12ab-4069-a36f-c777cf690d6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=90c0d211518e4254acebf59e345921ea, user_id=f9781166ed354aa2b36dfa2e90c53617, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:16.085093 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5520d68e-12ab-4069-a36f-c777cf690d6d admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:16.095359 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1069/2118] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:15 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:16.099728 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c8a52-5204-4c7d-b6c1-a26b0d119a93 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:16.099970 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1050/2119] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:16 2022] PUT /identity/v3/projects/90c0d211518e4254acebf59e345921ea/users/f9781166ed354aa2b36dfa2e90c53617/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:06:16.110975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899786c3-471b-4ca0-ad11-1665e9bdd21a None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:16.111224 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899786c3-471b-4ca0-ad11-1665e9bdd21a None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:16.111466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899786c3-471b-4ca0-ad11-1665e9bdd21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:16.140451 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-899786c3-471b-4ca0-ad11-1665e9bdd21a None None] MFA Rules not processed for user `f9781166ed354aa2b36dfa2e90c53617`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:16.141688 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-899786c3-471b-4ca0-ad11-1665e9bdd21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:16.182430 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c8a52-5204-4c7d-b6c1-a26b0d119a93 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:16.187832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66409b1-a94b-48b5-9e89-85201d110d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:16.188714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66409b1-a94b-48b5-9e89-85201d110d44 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:16.189230 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66409b1-a94b-48b5-9e89-85201d110d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:16.189579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66409b1-a94b-48b5-9e89-85201d110d44 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:16.190444 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66409b1-a94b-48b5-9e89-85201d110d44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:16.192731 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66409b1-a94b-48b5-9e89-85201d110d44 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:16.214227 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1070/2120] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:16 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:06:16.217607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66409b1-a94b-48b5-9e89-85201d110d44 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:16.225962 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1051/2121] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:06:16.230107 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66409b1-a94b-48b5-9e89-85201d110d44 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:16.238275 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8395c-0351-445a-82de-e0dc2973049d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:16.238962 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8395c-0351-445a-82de-e0dc2973049d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:16.239215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899786c3-471b-4ca0-ad11-1665e9bdd21a None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:16.239326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8395c-0351-445a-82de-e0dc2973049d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:16.239515 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8395c-0351-445a-82de-e0dc2973049d admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:16.240247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e8395c-0351-445a-82de-e0dc2973049d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:16.240684 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-899786c3-471b-4ca0-ad11-1665e9bdd21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:16.241886 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e8395c-0351-445a-82de-e0dc2973049d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:16.254572 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:16.258264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1071/2122] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:16.259204 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8395c-0351-445a-82de-e0dc2973049d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:16.271944 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8395c-0351-445a-82de-e0dc2973049d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:16.277340 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c958968-2d95-498f-92df-ddfb61c4a21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:16.278024 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c958968-2d95-498f-92df-ddfb61c4a21c admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:16.278584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c958968-2d95-498f-92df-ddfb61c4a21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:16.278727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c958968-2d95-498f-92df-ddfb61c4a21c admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:16.279348 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c958968-2d95-498f-92df-ddfb61c4a21c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:16.281365 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c958968-2d95-498f-92df-ddfb61c4a21c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:16.287598 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:16.295351 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1072/2123] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:06:16.300125 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c958968-2d95-498f-92df-ddfb61c4a21c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:16.317983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c958968-2d95-498f-92df-ddfb61c4a21c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:16.318283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5df3a-5a9e-45fa-9fdc-bd4825609a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:16.319264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5df3a-5a9e-45fa-9fdc-bd4825609a9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:16.319613 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5df3a-5a9e-45fa-9fdc-bd4825609a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:16.319917 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5df3a-5a9e-45fa-9fdc-bd4825609a9c admin admin] PATH_INFO: `/v3/projects/c8991d8cdbf84f74906e22418e8de36e/users/84e075c922674e26bd7742e42f70f0a1/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:16.324669 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899786c3-471b-4ca0-ad11-1665e9bdd21a None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:16.329351 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615bd11-405b-466f-9b88-5e332a68d751 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:16.330148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615bd11-405b-466f-9b88-5e332a68d751 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:16.330478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615bd11-405b-466f-9b88-5e332a68d751 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:16.330759 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615bd11-405b-466f-9b88-5e332a68d751 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:16.339262 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5df3a-5a9e-45fa-9fdc-bd4825609a9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8991d8cdbf84f74906e22418e8de36e, user_id=84e075c922674e26bd7742e42f70f0a1, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:16.342256 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5df3a-5a9e-45fa-9fdc-bd4825609a9c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:16.343544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615bd11-405b-466f-9b88-5e332a68d751 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:16.345998 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615bd11-405b-466f-9b88-5e332a68d751 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:16.359346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1073/2124] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:16 2022] PUT /identity/v3/projects/c8991d8cdbf84f74906e22418e8de36e/users/84e075c922674e26bd7742e42f70f0a1/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:06:16.360633 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1052/2125] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:16.381117 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379c585-1355-41af-8c30-981cf86c8c00 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:16.381956 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379c585-1355-41af-8c30-981cf86c8c00 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:16.382325 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379c585-1355-41af-8c30-981cf86c8c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:16.414528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3379c585-1355-41af-8c30-981cf86c8c00 None None] MFA Rules not processed for user `84e075c922674e26bd7742e42f70f0a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:16.415968 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3379c585-1355-41af-8c30-981cf86c8c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:16.499458 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1074/2126] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:16.504005 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615bd11-405b-466f-9b88-5e332a68d751 service cinder] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:16.515214 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615bd11-405b-466f-9b88-5e332a68d751 service cinder] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:16.584194 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9ba26-c4f6-495a-b574-4cf6833bd28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:16.585021 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9ba26-c4f6-495a-b574-4cf6833bd28b admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:16.585313 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9ba26-c4f6-495a-b574-4cf6833bd28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:16.585594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9ba26-c4f6-495a-b574-4cf6833bd28b admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:16.586409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9ba26-c4f6-495a-b574-4cf6833bd28b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:16.588350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9ba26-c4f6-495a-b574-4cf6833bd28b admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:16.617457 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1053/2127] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:16 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:16.622979 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379c585-1355-41af-8c30-981cf86c8c00 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:16.634210 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379c585-1355-41af-8c30-981cf86c8c00 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:16.751029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109d8cf-437a-43dd-a002-b918d7b71f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:16.753321 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109d8cf-437a-43dd-a002-b918d7b71f4d admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:16.754050 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109d8cf-437a-43dd-a002-b918d7b71f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:16.754572 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109d8cf-437a-43dd-a002-b918d7b71f4d admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:16.755798 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4109d8cf-437a-43dd-a002-b918d7b71f4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:16.760126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4109d8cf-437a-43dd-a002-b918d7b71f4d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:16.765488 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9ba26-c4f6-495a-b574-4cf6833bd28b admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:16.768291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-09a9ba26-c4f6-495a-b574-4cf6833bd28b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:16.794582 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9ba26-c4f6-495a-b574-4cf6833bd28b admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:16.802633 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1075/2128] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:16 2022] POST /identity/v3/users => generated 298 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:06:16.809655 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109d8cf-437a-43dd-a002-b918d7b71f4d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:16.809990 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f572ec7-5b1a-462c-8f99-b9a73890d31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:16.812871 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f572ec7-5b1a-462c-8f99-b9a73890d31f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:16.813638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f572ec7-5b1a-462c-8f99-b9a73890d31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:16.826810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f572ec7-5b1a-462c-8f99-b9a73890d31f admin admin] PATH_INFO: `/v3/users/4515a88c5876460f8d70aff9f7ca6875` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:16.826810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f572ec7-5b1a-462c-8f99-b9a73890d31f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4515a88c5876460f8d70aff9f7ca6875)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:16.826810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f572ec7-5b1a-462c-8f99-b9a73890d31f admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:16.842407 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109d8cf-437a-43dd-a002-b918d7b71f4d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:16.853315 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce257f-6095-4fb7-87ba-d98f1599e2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:16.854413 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce257f-6095-4fb7-87ba-d98f1599e2b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:16.854866 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce257f-6095-4fb7-87ba-d98f1599e2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:16.855270 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce257f-6095-4fb7-87ba-d98f1599e2b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:16.856595 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce257f-6095-4fb7-87ba-d98f1599e2b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:16.859141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce257f-6095-4fb7-87ba-d98f1599e2b6 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:16.876742 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1076/2129] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:16.882026 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce257f-6095-4fb7-87ba-d98f1599e2b6 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:16.888932 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2f572ec7-5b1a-462c-8f99-b9a73890d31f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4515a88c5876460f8d70aff9f7ca6875'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:16.900496 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce257f-6095-4fb7-87ba-d98f1599e2b6 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:16.904594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2f572ec7-5b1a-462c-8f99-b9a73890d31f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4515a88c5876460f8d70aff9f7ca6875'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:16.906276 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b462a2-986a-4f1e-ab53-0cbc7f1111be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:16.906475 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2f572ec7-5b1a-462c-8f99-b9a73890d31f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4515a88c5876460f8d70aff9f7ca6875'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:16.907521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b462a2-986a-4f1e-ab53-0cbc7f1111be admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:16.908013 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2f572ec7-5b1a-462c-8f99-b9a73890d31f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4515a88c5876460f8d70aff9f7ca6875'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:16.908098 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b462a2-986a-4f1e-ab53-0cbc7f1111be admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:16.908098 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b462a2-986a-4f1e-ab53-0cbc7f1111be admin admin] PATH_INFO: `/v3/projects/0ffd96dad7224169926e7d2d8c2213e9/users/839c45ab51a04dafb97972c31ef6a001/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:16.919879 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2f572ec7-5b1a-462c-8f99-b9a73890d31f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4515a88c5876460f8d70aff9f7ca6875'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:16.928634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b462a2-986a-4f1e-ab53-0cbc7f1111be admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ffd96dad7224169926e7d2d8c2213e9, user_id=839c45ab51a04dafb97972c31ef6a001, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:16.931460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b462a2-986a-4f1e-ab53-0cbc7f1111be admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:16.960760 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1077/2130] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:16 2022] PUT /identity/v3/projects/0ffd96dad7224169926e7d2d8c2213e9/users/839c45ab51a04dafb97972c31ef6a001/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:06:16.960973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1054/2131] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:16 2022] DELETE /identity/v3/users/4515a88c5876460f8d70aff9f7ca6875 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:16.961131 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f717a5-2605-47fb-913c-b954372afc2f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:16.961418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f717a5-2605-47fb-913c-b954372afc2f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:16.961656 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f717a5-2605-47fb-913c-b954372afc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:16.965018 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f572ec7-5b1a-462c-8f99-b9a73890d31f admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:16.966613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-2f572ec7-5b1a-462c-8f99-b9a73890d31f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:16.985190 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f4f717a5-2605-47fb-913c-b954372afc2f None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:16.988048 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f4f717a5-2605-47fb-913c-b954372afc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:16.993517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:17.012627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:17.090077 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f572ec7-5b1a-462c-8f99-b9a73890d31f admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:17.091711 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1078/2132] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:17.095996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d334f3-56b3-449a-9394-ac39121c2bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:17.096911 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d334f3-56b3-449a-9394-ac39121c2bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:17.097308 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d334f3-56b3-449a-9394-ac39121c2bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:17.097586 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d334f3-56b3-449a-9394-ac39121c2bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:17.098888 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d334f3-56b3-449a-9394-ac39121c2bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:17.100064 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f717a5-2605-47fb-913c-b954372afc2f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:17.100553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d334f3-56b3-449a-9394-ac39121c2bd8 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:17.103170 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f4f717a5-2605-47fb-913c-b954372afc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:17.114144 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1055/2133] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:17.123368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d334f3-56b3-449a-9394-ac39121c2bd8 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:17.136687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d334f3-56b3-449a-9394-ac39121c2bd8 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:17.144094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f61451-dc50-42a3-9a61-8200ffb9f268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:17.145728 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f61451-dc50-42a3-9a61-8200ffb9f268 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:17.146840 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f61451-dc50-42a3-9a61-8200ffb9f268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:17.147829 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f61451-dc50-42a3-9a61-8200ffb9f268 admin admin] PATH_INFO: `/v3/projects/0ffd96dad7224169926e7d2d8c2213e9/users/839c45ab51a04dafb97972c31ef6a001/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:17.150130 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f61451-dc50-42a3-9a61-8200ffb9f268 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ffd96dad7224169926e7d2d8c2213e9, user_id=839c45ab51a04dafb97972c31ef6a001, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:17.154430 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f61451-dc50-42a3-9a61-8200ffb9f268 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:17.174955 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1056/2134] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:17 2022] PUT /identity/v3/projects/0ffd96dad7224169926e7d2d8c2213e9/users/839c45ab51a04dafb97972c31ef6a001/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:17.187401 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f61451-dc50-42a3-9a61-8200ffb9f268 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:17.187401 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-52f61451-dc50-42a3-9a61-8200ffb9f268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:17.242006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f717a5-2605-47fb-913c-b954372afc2f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:17.265288 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f61451-dc50-42a3-9a61-8200ffb9f268 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:17.269469 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88a89c-8099-49ee-88f7-dfe96c6a2c1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:17.270321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88a89c-8099-49ee-88f7-dfe96c6a2c1c service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:17.270611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88a89c-8099-49ee-88f7-dfe96c6a2c1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:17.270898 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88a89c-8099-49ee-88f7-dfe96c6a2c1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:17.273315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-dc88a89c-8099-49ee-88f7-dfe96c6a2c1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:17.316768 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790f8fb-9197-425c-bc30-e61248631ee9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:17.318084 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790f8fb-9197-425c-bc30-e61248631ee9 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:17.318447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790f8fb-9197-425c-bc30-e61248631ee9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:17.318791 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790f8fb-9197-425c-bc30-e61248631ee9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:17.355654 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88a89c-8099-49ee-88f7-dfe96c6a2c1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:17.357255 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88a89c-8099-49ee-88f7-dfe96c6a2c1c service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:17.370282 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1057/2135] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:17.371809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e656845-530b-4f52-bb42-b429373b21a8 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:17.372158 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e656845-530b-4f52-bb42-b429373b21a8 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:17.372529 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e656845-530b-4f52-bb42-b429373b21a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:17.418561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9790f8fb-9197-425c-bc30-e61248631ee9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:17.420225 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9790f8fb-9197-425c-bc30-e61248631ee9 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:17.437882 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1079/2136] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:06:17.442775 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-8e656845-530b-4f52-bb42-b429373b21a8 None None] MFA Rules not processed for user `839c45ab51a04dafb97972c31ef6a001`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:17.444368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-8e656845-530b-4f52-bb42-b429373b21a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:17.504557 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:17.591986 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1058/2137] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:17.616247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790f8fb-9197-425c-bc30-e61248631ee9 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:17.618143 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9790f8fb-9197-425c-bc30-e61248631ee9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:17.638735 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790f8fb-9197-425c-bc30-e61248631ee9 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:17.644454 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1cf276-9742-48bd-9703-f8ac09f6df55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:17.645845 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cf276-9742-48bd-9703-f8ac09f6df55 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:17.646138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cf276-9742-48bd-9703-f8ac09f6df55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:17.646453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cf276-9742-48bd-9703-f8ac09f6df55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:17.659859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1cf276-9742-48bd-9703-f8ac09f6df55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:17.661944 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1cf276-9742-48bd-9703-f8ac09f6df55 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:17.673982 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1080/2138] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:17 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:17.890868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e656845-530b-4f52-bb42-b429373b21a8 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:17.892710 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-8e656845-530b-4f52-bb42-b429373b21a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:17.929174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e656845-530b-4f52-bb42-b429373b21a8 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:17.935466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c6caa-9f25-4870-814d-28b6016ce65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:17.937449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c6caa-9f25-4870-814d-28b6016ce65c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:17.937880 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c6caa-9f25-4870-814d-28b6016ce65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:17.938375 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c6caa-9f25-4870-814d-28b6016ce65c admin admin] PATH_INFO: `/v3/projects/eaea742a13bd43ba88339d0d1095dbb3` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:17.939882 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c6caa-9f25-4870-814d-28b6016ce65c admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaea742a13bd43ba88339d0d1095dbb3)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:17.943046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c6caa-9f25-4870-814d-28b6016ce65c admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:17.975971 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1cf276-9742-48bd-9703-f8ac09f6df55 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:17.995395 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1cf276-9742-48bd-9703-f8ac09f6df55 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:18.002036 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ad91d-addf-4a68-aa04-1c5a317d8c37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:18.003045 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ad91d-addf-4a68-aa04-1c5a317d8c37 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:18.003514 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ad91d-addf-4a68-aa04-1c5a317d8c37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:18.003945 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ad91d-addf-4a68-aa04-1c5a317d8c37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:18.006178 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-835ad91d-addf-4a68-aa04-1c5a317d8c37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:18.021727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:18.043274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-261c6caa-9f25-4870-814d-28b6016ce65c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaea742a13bd43ba88339d0d1095dbb3', 'request_id': 'req-261c6caa-9f25-4870-814d-28b6016ce65c'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:18.056671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-261c6caa-9f25-4870-814d-28b6016ce65c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaea742a13bd43ba88339d0d1095dbb3', 'request_id': 'req-261c6caa-9f25-4870-814d-28b6016ce65c'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:18.071799 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-261c6caa-9f25-4870-814d-28b6016ce65c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaea742a13bd43ba88339d0d1095dbb3', 'request_id': 'req-261c6caa-9f25-4870-814d-28b6016ce65c'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:18.087503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-261c6caa-9f25-4870-814d-28b6016ce65c admin admin] The token cache is being invalidate because project eaea742a13bd43ba88339d0d1095dbb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:18.087791 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-261c6caa-9f25-4870-814d-28b6016ce65c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:18.088468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-261c6caa-9f25-4870-814d-28b6016ce65c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:18.090567 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1059/2139] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:17 2022] DELETE /identity/v3/projects/eaea742a13bd43ba88339d0d1095dbb3 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:18.094935 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c6caa-9f25-4870-814d-28b6016ce65c admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:18.096718 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-261c6caa-9f25-4870-814d-28b6016ce65c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:18.110332 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ad91d-addf-4a68-aa04-1c5a317d8c37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:18.111541 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ad91d-addf-4a68-aa04-1c5a317d8c37 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:18.135410 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1081/2140] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:18.195727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c6caa-9f25-4870-814d-28b6016ce65c admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:18.200697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41f47b-3738-4f37-bbfb-abbb7fb607c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:18.201596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41f47b-3738-4f37-bbfb-abbb7fb607c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:18.201879 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41f47b-3738-4f37-bbfb-abbb7fb607c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:18.202250 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41f47b-3738-4f37-bbfb-abbb7fb607c9 admin admin] PATH_INFO: `/v3/users/89db5667f0c145878eff188e22352660` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:18.205018 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41f47b-3738-4f37-bbfb-abbb7fb607c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89db5667f0c145878eff188e22352660)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:18.206995 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41f47b-3738-4f37-bbfb-abbb7fb607c9 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:18.278967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-dd41f47b-3738-4f37-bbfb-abbb7fb607c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89db5667f0c145878eff188e22352660'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:18.289765 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-dd41f47b-3738-4f37-bbfb-abbb7fb607c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89db5667f0c145878eff188e22352660'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:18.290743 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-dd41f47b-3738-4f37-bbfb-abbb7fb607c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89db5667f0c145878eff188e22352660'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:18.291400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-dd41f47b-3738-4f37-bbfb-abbb7fb607c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89db5667f0c145878eff188e22352660'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:18.294860 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:18.305684 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-dd41f47b-3738-4f37-bbfb-abbb7fb607c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89db5667f0c145878eff188e22352660'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:18.327473 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1060/2141] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:18 2022] DELETE /identity/v3/users/89db5667f0c145878eff188e22352660 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:18.676904 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ad91d-addf-4a68-aa04-1c5a317d8c37 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:18.691818 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ad91d-addf-4a68-aa04-1c5a317d8c37 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:18.775206 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f685e-b789-4617-bb65-e62228adb418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:18.776699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f685e-b789-4617-bb65-e62228adb418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:18.776699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f685e-b789-4617-bb65-e62228adb418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:18.776904 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f685e-b789-4617-bb65-e62228adb418 admin admin] PATH_INFO: `/v3/projects/66edceff07334667985a940cd9cc9bfe` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:18.778211 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f685e-b789-4617-bb65-e62228adb418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66edceff07334667985a940cd9cc9bfe)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:18.780598 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f685e-b789-4617-bb65-e62228adb418 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:18.850805 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ee4f685e-b789-4617-bb65-e62228adb418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66edceff07334667985a940cd9cc9bfe', 'request_id': 'req-ee4f685e-b789-4617-bb65-e62228adb418'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:18.857487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ee4f685e-b789-4617-bb65-e62228adb418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66edceff07334667985a940cd9cc9bfe', 'request_id': 'req-ee4f685e-b789-4617-bb65-e62228adb418'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:18.867650 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ee4f685e-b789-4617-bb65-e62228adb418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66edceff07334667985a940cd9cc9bfe', 'request_id': 'req-ee4f685e-b789-4617-bb65-e62228adb418'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:18.886526 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ee4f685e-b789-4617-bb65-e62228adb418 admin admin] The token cache is being invalidate because project 66edceff07334667985a940cd9cc9bfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:18.886726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ee4f685e-b789-4617-bb65-e62228adb418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:18.887401 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ee4f685e-b789-4617-bb65-e62228adb418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:18.891969 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1082/2142] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:18 2022] DELETE /identity/v3/projects/66edceff07334667985a940cd9cc9bfe => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:18.897687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41f47b-3738-4f37-bbfb-abbb7fb607c9 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:18.900849 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-dd41f47b-3738-4f37-bbfb-abbb7fb607c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:18.933270 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41f47b-3738-4f37-bbfb-abbb7fb607c9 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:19.017945 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e124fa0-3553-4a74-93c4-2643924bf6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:19.018885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e124fa0-3553-4a74-93c4-2643924bf6f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:19.019324 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e124fa0-3553-4a74-93c4-2643924bf6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:19.019795 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e124fa0-3553-4a74-93c4-2643924bf6f6 admin admin] PATH_INFO: `/v3/users/2bf6d92d019642fa8e40a5e1fd66f01c` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:19.021170 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e124fa0-3553-4a74-93c4-2643924bf6f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bf6d92d019642fa8e40a5e1fd66f01c)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:19.023347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e124fa0-3553-4a74-93c4-2643924bf6f6 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:19.098810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1e124fa0-3553-4a74-93c4-2643924bf6f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bf6d92d019642fa8e40a5e1fd66f01c'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:19.109726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1e124fa0-3553-4a74-93c4-2643924bf6f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bf6d92d019642fa8e40a5e1fd66f01c'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:19.110578 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1e124fa0-3553-4a74-93c4-2643924bf6f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bf6d92d019642fa8e40a5e1fd66f01c'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:19.111402 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1e124fa0-3553-4a74-93c4-2643924bf6f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bf6d92d019642fa8e40a5e1fd66f01c'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:19.121279 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-1e124fa0-3553-4a74-93c4-2643924bf6f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bf6d92d019642fa8e40a5e1fd66f01c'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:19.146246 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1061/2143] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:18 2022] DELETE /identity/v3/users/2bf6d92d019642fa8e40a5e1fd66f01c => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:19.551044 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f685e-b789-4617-bb65-e62228adb418 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:19.554134 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ee4f685e-b789-4617-bb65-e62228adb418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:19.586537 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f685e-b789-4617-bb65-e62228adb418 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:19.654393 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:19.685845 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a7f55c-0f75-45cc-b309-09328479f30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:19.687390 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7f55c-0f75-45cc-b309-09328479f30b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:19.687962 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7f55c-0f75-45cc-b309-09328479f30b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:19.688325 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7f55c-0f75-45cc-b309-09328479f30b admin admin] PATH_INFO: `/v3/projects/f1a7e81034eb439591a666ad30796c38` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:19.690600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a7f55c-0f75-45cc-b309-09328479f30b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1a7e81034eb439591a666ad30796c38)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:19.694435 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a7f55c-0f75-45cc-b309-09328479f30b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:19.784956 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c6a7f55c-0f75-45cc-b309-09328479f30b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1a7e81034eb439591a666ad30796c38', 'request_id': 'req-c6a7f55c-0f75-45cc-b309-09328479f30b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:19.792178 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c6a7f55c-0f75-45cc-b309-09328479f30b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1a7e81034eb439591a666ad30796c38', 'request_id': 'req-c6a7f55c-0f75-45cc-b309-09328479f30b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:19.804061 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c6a7f55c-0f75-45cc-b309-09328479f30b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1a7e81034eb439591a666ad30796c38', 'request_id': 'req-c6a7f55c-0f75-45cc-b309-09328479f30b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:19.825202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c6a7f55c-0f75-45cc-b309-09328479f30b admin admin] The token cache is being invalidate because project f1a7e81034eb439591a666ad30796c38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:19.825627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c6a7f55c-0f75-45cc-b309-09328479f30b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:19.826761 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c6a7f55c-0f75-45cc-b309-09328479f30b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:19.830042 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1083/2144] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:19 2022] DELETE /identity/v3/projects/f1a7e81034eb439591a666ad30796c38 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:19.837727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd2d81-9f49-4d6e-a6ca-54bbec4d8585 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:19.838243 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd2d81-9f49-4d6e-a6ca-54bbec4d8585 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:19.838713 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd2d81-9f49-4d6e-a6ca-54bbec4d8585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:19.872404 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-22dd2d81-9f49-4d6e-a6ca-54bbec4d8585 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:19.873727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-22dd2d81-9f49-4d6e-a6ca-54bbec4d8585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:19.933544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:19.954242 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1062/2145] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:19.962487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17e900-24c6-46b1-95ac-5d3b1a1d0799 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:19.962890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17e900-24c6-46b1-95ac-5d3b1a1d0799 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:19.963285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17e900-24c6-46b1-95ac-5d3b1a1d0799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:19.987781 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3a17e900-24c6-46b1-95ac-5d3b1a1d0799 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:19.989022 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3a17e900-24c6-46b1-95ac-5d3b1a1d0799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:19.997629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1084/2146] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:20.002866 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd2d81-9f49-4d6e-a6ca-54bbec4d8585 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:20.012965 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd2d81-9f49-4d6e-a6ca-54bbec4d8585 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:20.018795 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921ef31-6aea-4a2f-aabe-5df7171e2c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:20.019477 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921ef31-6aea-4a2f-aabe-5df7171e2c74 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:20.019763 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921ef31-6aea-4a2f-aabe-5df7171e2c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:20.020040 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921ef31-6aea-4a2f-aabe-5df7171e2c74 admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:20.020773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921ef31-6aea-4a2f-aabe-5df7171e2c74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:06:20.021090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921ef31-6aea-4a2f-aabe-5df7171e2c74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:20.022770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921ef31-6aea-4a2f-aabe-5df7171e2c74 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:20.035976 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1063/2147] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:06:20.043118 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17e900-24c6-46b1-95ac-5d3b1a1d0799 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:20.060554 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17e900-24c6-46b1-95ac-5d3b1a1d0799 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:20.069802 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad596da0-ae89-4d93-a135-a854113525f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:20.071036 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad596da0-ae89-4d93-a135-a854113525f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:20.071549 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad596da0-ae89-4d93-a135-a854113525f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:20.072043 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad596da0-ae89-4d93-a135-a854113525f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:20.073369 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad596da0-ae89-4d93-a135-a854113525f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:20.078024 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad596da0-ae89-4d93-a135-a854113525f6 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:20.118001 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1085/2148] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:20 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:20.124610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921ef31-6aea-4a2f-aabe-5df7171e2c74 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:20.135373 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921ef31-6aea-4a2f-aabe-5df7171e2c74 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:20.204918 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f72e1-ba81-4edb-8fd8-323d4cc3d2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:20.205609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f72e1-ba81-4edb-8fd8-323d4cc3d2b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:20.205909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f72e1-ba81-4edb-8fd8-323d4cc3d2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:20.206167 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f72e1-ba81-4edb-8fd8-323d4cc3d2b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:20.206911 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40f72e1-ba81-4edb-8fd8-323d4cc3d2b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:20.208856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40f72e1-ba81-4edb-8fd8-323d4cc3d2b2 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:20.244585 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1064/2149] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:20 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:20.245855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad596da0-ae89-4d93-a135-a854113525f6 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:20.258358 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad596da0-ae89-4d93-a135-a854113525f6 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:20.263955 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93281f1d-459b-4660-a268-f61c3553920d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:20.264715 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93281f1d-459b-4660-a268-f61c3553920d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:20.265836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93281f1d-459b-4660-a268-f61c3553920d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:20.265986 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93281f1d-459b-4660-a268-f61c3553920d admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:20.266665 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93281f1d-459b-4660-a268-f61c3553920d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:20.268284 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93281f1d-459b-4660-a268-f61c3553920d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:20.281683 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1086/2150] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:20.286529 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f72e1-ba81-4edb-8fd8-323d4cc3d2b2 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:20.301245 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f72e1-ba81-4edb-8fd8-323d4cc3d2b2 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:20.306898 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c13bb-a1ba-4bfb-86e6-4923335ca3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:20.307628 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c13bb-a1ba-4bfb-86e6-4923335ca3c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:20.307919 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c13bb-a1ba-4bfb-86e6-4923335ca3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:20.308228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c13bb-a1ba-4bfb-86e6-4923335ca3c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:20.308972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c13bb-a1ba-4bfb-86e6-4923335ca3c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:20.310681 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c13bb-a1ba-4bfb-86e6-4923335ca3c8 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:20.325534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1065/2151] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:20.330541 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93281f1d-459b-4660-a268-f61c3553920d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:20.340990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93281f1d-459b-4660-a268-f61c3553920d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:20.346520 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84587933-d377-486a-a5a9-c0552db75fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:20.347419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84587933-d377-486a-a5a9-c0552db75fb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:20.347699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84587933-d377-486a-a5a9-c0552db75fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:20.347974 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84587933-d377-486a-a5a9-c0552db75fb4 admin admin] PATH_INFO: `/v3/projects/799f6c254b56426e89102354e6274b18/users/d9790a81d03741688e45948876714cb4/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:20.365346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84587933-d377-486a-a5a9-c0552db75fb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=799f6c254b56426e89102354e6274b18, user_id=d9790a81d03741688e45948876714cb4, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:20.368237 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84587933-d377-486a-a5a9-c0552db75fb4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:20.383966 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1087/2152] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:20 2022] PUT /identity/v3/projects/799f6c254b56426e89102354e6274b18/users/d9790a81d03741688e45948876714cb4/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:20.395384 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7bb49-022a-4504-b35c-4ee2500da83d None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:20.395676 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7bb49-022a-4504-b35c-4ee2500da83d None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:20.395957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7bb49-022a-4504-b35c-4ee2500da83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:20.426126 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-77b7bb49-022a-4504-b35c-4ee2500da83d None None] MFA Rules not processed for user `d9790a81d03741688e45948876714cb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:20.427086 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-77b7bb49-022a-4504-b35c-4ee2500da83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:20.529611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1066/2153] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:20.541554 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84587933-d377-486a-a5a9-c0552db75fb4 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:20.543853 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-84587933-d377-486a-a5a9-c0552db75fb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:20.628406 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84587933-d377-486a-a5a9-c0552db75fb4 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:20.632995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e3626-c9ab-4597-a881-8cdc7384d993 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:20.634093 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e3626-c9ab-4597-a881-8cdc7384d993 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:20.634616 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e3626-c9ab-4597-a881-8cdc7384d993 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:20.635143 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e3626-c9ab-4597-a881-8cdc7384d993 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:20.651195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e3626-c9ab-4597-a881-8cdc7384d993 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:20.652831 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e3626-c9ab-4597-a881-8cdc7384d993 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:20.664562 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1088/2154] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:06:26.368102 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7ba62-40fa-4f22-8913-3eab0362c950 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:26.368868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7ba62-40fa-4f22-8913-3eab0362c950 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:26.368868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7ba62-40fa-4f22-8913-3eab0362c950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:26.401144 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-6ad7ba62-40fa-4f22-8913-3eab0362c950 None None] MFA Rules not processed for user `d9790a81d03741688e45948876714cb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:26.403246 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-6ad7ba62-40fa-4f22-8913-3eab0362c950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:26.418297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1067/2155] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:27.342270 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e3626-c9ab-4597-a881-8cdc7384d993 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:27.345476 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-703e3626-c9ab-4597-a881-8cdc7384d993 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:27.370070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:27.461254 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e3626-c9ab-4597-a881-8cdc7384d993 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:27.466550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd768e7-6a23-4a60-a03d-79f31d07fb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:27.468361 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd768e7-6a23-4a60-a03d-79f31d07fb4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:27.468667 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd768e7-6a23-4a60-a03d-79f31d07fb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:27.468950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd768e7-6a23-4a60-a03d-79f31d07fb4b admin admin] PATH_INFO: `/v3/users/84e075c922674e26bd7742e42f70f0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:27.470378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd768e7-6a23-4a60-a03d-79f31d07fb4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=84e075c922674e26bd7742e42f70f0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:27.472372 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd768e7-6a23-4a60-a03d-79f31d07fb4b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:27.540651 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ecd768e7-6a23-4a60-a03d-79f31d07fb4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84e075c922674e26bd7742e42f70f0a1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:27.553569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ecd768e7-6a23-4a60-a03d-79f31d07fb4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84e075c922674e26bd7742e42f70f0a1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:27.570544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ecd768e7-6a23-4a60-a03d-79f31d07fb4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84e075c922674e26bd7742e42f70f0a1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:27.571551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ecd768e7-6a23-4a60-a03d-79f31d07fb4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84e075c922674e26bd7742e42f70f0a1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:27.572403 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ecd768e7-6a23-4a60-a03d-79f31d07fb4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84e075c922674e26bd7742e42f70f0a1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:27.593743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1089/2156] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:27 2022] DELETE /identity/v3/users/84e075c922674e26bd7742e42f70f0a1 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:27.601224 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecc96a-9152-4851-a698-ddd41bc86712 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:27.601840 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecc96a-9152-4851-a698-ddd41bc86712 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:27.602060 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecc96a-9152-4851-a698-ddd41bc86712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:27.634391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-22ecc96a-9152-4851-a698-ddd41bc86712 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:27.636743 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-22ecc96a-9152-4851-a698-ddd41bc86712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:27.704164 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:27.736316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1068/2157] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:27.743168 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd768e7-6a23-4a60-a03d-79f31d07fb4b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:27.744709 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ecd768e7-6a23-4a60-a03d-79f31d07fb4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:27.841121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd768e7-6a23-4a60-a03d-79f31d07fb4b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:27.846403 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7511650-2b31-47a5-a45c-8953b2af305e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:27.847086 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7511650-2b31-47a5-a45c-8953b2af305e service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:27.847405 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7511650-2b31-47a5-a45c-8953b2af305e service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:27.847653 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7511650-2b31-47a5-a45c-8953b2af305e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:27.862030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7511650-2b31-47a5-a45c-8953b2af305e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:27.863927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7511650-2b31-47a5-a45c-8953b2af305e service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:27.881093 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1090/2158] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:06:28.296043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecc96a-9152-4851-a698-ddd41bc86712 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:28.297772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-22ecc96a-9152-4851-a698-ddd41bc86712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:28.439749 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecc96a-9152-4851-a698-ddd41bc86712 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:28.449820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3dc74-73fd-4d9a-a549-d57744686d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:28.451061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3dc74-73fd-4d9a-a549-d57744686d4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:28.451707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3dc74-73fd-4d9a-a549-d57744686d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:28.453128 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3dc74-73fd-4d9a-a549-d57744686d4d admin admin] PATH_INFO: `/v3/projects/c8991d8cdbf84f74906e22418e8de36e` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:28.455264 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3dc74-73fd-4d9a-a549-d57744686d4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8991d8cdbf84f74906e22418e8de36e)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:28.459356 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3dc74-73fd-4d9a-a549-d57744686d4d admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:28.494785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7511650-2b31-47a5-a45c-8953b2af305e service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:28.496834 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d7511650-2b31-47a5-a45c-8953b2af305e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:28.570926 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:28.600476 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-b3d3dc74-73fd-4d9a-a549-d57744686d4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8991d8cdbf84f74906e22418e8de36e', 'request_id': 'req-b3d3dc74-73fd-4d9a-a549-d57744686d4d'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:28.615288 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-b3d3dc74-73fd-4d9a-a549-d57744686d4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8991d8cdbf84f74906e22418e8de36e', 'request_id': 'req-b3d3dc74-73fd-4d9a-a549-d57744686d4d'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:28.624937 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-b3d3dc74-73fd-4d9a-a549-d57744686d4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8991d8cdbf84f74906e22418e8de36e', 'request_id': 'req-b3d3dc74-73fd-4d9a-a549-d57744686d4d'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:28.639900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-b3d3dc74-73fd-4d9a-a549-d57744686d4d admin admin] The token cache is being invalidate because project c8991d8cdbf84f74906e22418e8de36e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:28.640463 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-b3d3dc74-73fd-4d9a-a549-d57744686d4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:28.641475 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-b3d3dc74-73fd-4d9a-a549-d57744686d4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:28.643351 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1069/2159] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:28 2022] DELETE /identity/v3/projects/c8991d8cdbf84f74906e22418e8de36e => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:28.647479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3dc74-73fd-4d9a-a549-d57744686d4d admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:28.649874 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-b3d3dc74-73fd-4d9a-a549-d57744686d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:28.674787 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7511650-2b31-47a5-a45c-8953b2af305e service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:28.785891 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade87c3-8391-4e0f-ae68-e1570b3d0a7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:28.789247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade87c3-8391-4e0f-ae68-e1570b3d0a7b service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:28.790595 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade87c3-8391-4e0f-ae68-e1570b3d0a7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:28.792046 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade87c3-8391-4e0f-ae68-e1570b3d0a7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:28.795279 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bade87c3-8391-4e0f-ae68-e1570b3d0a7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:28.798016 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3dc74-73fd-4d9a-a549-d57744686d4d admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:28.805879 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5237bcb-ba72-49a1-88db-e78b653f58ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:28.807127 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5237bcb-ba72-49a1-88db-e78b653f58ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:28.808169 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5237bcb-ba72-49a1-88db-e78b653f58ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:28.808742 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5237bcb-ba72-49a1-88db-e78b653f58ed admin admin] PATH_INFO: `/v3/users/839c45ab51a04dafb97972c31ef6a001` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:28.813272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5237bcb-ba72-49a1-88db-e78b653f58ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=839c45ab51a04dafb97972c31ef6a001)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:28.814626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5237bcb-ba72-49a1-88db-e78b653f58ed admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:28.821615 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:28.893016 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d5237bcb-ba72-49a1-88db-e78b653f58ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '839c45ab51a04dafb97972c31ef6a001'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:28.905605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d5237bcb-ba72-49a1-88db-e78b653f58ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '839c45ab51a04dafb97972c31ef6a001'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:28.907293 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d5237bcb-ba72-49a1-88db-e78b653f58ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '839c45ab51a04dafb97972c31ef6a001'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:28.908217 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d5237bcb-ba72-49a1-88db-e78b653f58ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '839c45ab51a04dafb97972c31ef6a001'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:28.909708 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade87c3-8391-4e0f-ae68-e1570b3d0a7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:28.911671 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade87c3-8391-4e0f-ae68-e1570b3d0a7b service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:28.917657 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d5237bcb-ba72-49a1-88db-e78b653f58ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '839c45ab51a04dafb97972c31ef6a001'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:28.924109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1091/2160] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:06:28.939013 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1070/2161] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:28 2022] DELETE /identity/v3/users/839c45ab51a04dafb97972c31ef6a001 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:29.434543 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade87c3-8391-4e0f-ae68-e1570b3d0a7b service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:29.454835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade87c3-8391-4e0f-ae68-e1570b3d0a7b service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:29.562360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68eabe-35d7-46d2-8cff-f81f835b1bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:29.564099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68eabe-35d7-46d2-8cff-f81f835b1bfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:29.564402 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68eabe-35d7-46d2-8cff-f81f835b1bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:29.564728 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68eabe-35d7-46d2-8cff-f81f835b1bfa admin admin] PATH_INFO: `/v3/projects/0ffd96dad7224169926e7d2d8c2213e9` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:29.569813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68eabe-35d7-46d2-8cff-f81f835b1bfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ffd96dad7224169926e7d2d8c2213e9)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:29.572598 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68eabe-35d7-46d2-8cff-f81f835b1bfa admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:29.624062 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:29.662653 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0b68eabe-35d7-46d2-8cff-f81f835b1bfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ffd96dad7224169926e7d2d8c2213e9', 'request_id': 'req-0b68eabe-35d7-46d2-8cff-f81f835b1bfa'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:29.672390 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0b68eabe-35d7-46d2-8cff-f81f835b1bfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ffd96dad7224169926e7d2d8c2213e9', 'request_id': 'req-0b68eabe-35d7-46d2-8cff-f81f835b1bfa'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:29.684196 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0b68eabe-35d7-46d2-8cff-f81f835b1bfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ffd96dad7224169926e7d2d8c2213e9', 'request_id': 'req-0b68eabe-35d7-46d2-8cff-f81f835b1bfa'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:29.706922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0b68eabe-35d7-46d2-8cff-f81f835b1bfa admin admin] The token cache is being invalidate because project 0ffd96dad7224169926e7d2d8c2213e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:29.707243 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0b68eabe-35d7-46d2-8cff-f81f835b1bfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:29.708366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0b68eabe-35d7-46d2-8cff-f81f835b1bfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:29.710428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1092/2162] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:29 2022] DELETE /identity/v3/projects/0ffd96dad7224169926e7d2d8c2213e9 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:29.715212 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25540c-f6c4-40ef-9993-9ced32a88083 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:29.715763 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25540c-f6c4-40ef-9993-9ced32a88083 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:29.715763 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25540c-f6c4-40ef-9993-9ced32a88083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:29.782290 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-9c25540c-f6c4-40ef-9993-9ced32a88083 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:29.784819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-9c25540c-f6c4-40ef-9993-9ced32a88083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:29.898237 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1071/2163] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:29.912144 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddfad9-5f72-4254-85dc-7be3335780ef None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:29.912410 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddfad9-5f72-4254-85dc-7be3335780ef None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:29.912952 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddfad9-5f72-4254-85dc-7be3335780ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:29.938885 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-74ddfad9-5f72-4254-85dc-7be3335780ef None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:29.940397 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-74ddfad9-5f72-4254-85dc-7be3335780ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:29.958559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1093/2164] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:29.962036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25540c-f6c4-40ef-9993-9ced32a88083 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:30.001768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25540c-f6c4-40ef-9993-9ced32a88083 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:30.016926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c150ed46-5e9f-4d95-94de-6d8034fe3051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:30.018154 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150ed46-5e9f-4d95-94de-6d8034fe3051 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:30.018668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150ed46-5e9f-4d95-94de-6d8034fe3051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:30.019386 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150ed46-5e9f-4d95-94de-6d8034fe3051 admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:30.021580 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c150ed46-5e9f-4d95-94de-6d8034fe3051 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:06:30.022381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c150ed46-5e9f-4d95-94de-6d8034fe3051 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:30.027100 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c150ed46-5e9f-4d95-94de-6d8034fe3051 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:30.059861 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1072/2165] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:06:30.064234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddfad9-5f72-4254-85dc-7be3335780ef None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:30.078764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddfad9-5f72-4254-85dc-7be3335780ef None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:30.084854 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef632c6e-0b3a-4eb1-95f3-31e4f3502969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:30.085780 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef632c6e-0b3a-4eb1-95f3-31e4f3502969 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:30.086095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef632c6e-0b3a-4eb1-95f3-31e4f3502969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:30.086504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef632c6e-0b3a-4eb1-95f3-31e4f3502969 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:30.087837 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef632c6e-0b3a-4eb1-95f3-31e4f3502969 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:30.089892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef632c6e-0b3a-4eb1-95f3-31e4f3502969 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:30.137069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1094/2166] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:30 2022] POST /identity/v3/projects => generated 387 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:06:30.140652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c150ed46-5e9f-4d95-94de-6d8034fe3051 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:30.141002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef632c6e-0b3a-4eb1-95f3-31e4f3502969 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:30.143486 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ef632c6e-0b3a-4eb1-95f3-31e4f3502969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:30.154695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c150ed46-5e9f-4d95-94de-6d8034fe3051 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:30.228314 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:30.255174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887088c8-2b35-47b2-aa37-096b59c502fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:30.256028 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887088c8-2b35-47b2-aa37-096b59c502fd admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:30.256209 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef632c6e-0b3a-4eb1-95f3-31e4f3502969 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:30.256523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887088c8-2b35-47b2-aa37-096b59c502fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:30.256935 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887088c8-2b35-47b2-aa37-096b59c502fd admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:30.257718 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887088c8-2b35-47b2-aa37-096b59c502fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:30.259989 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887088c8-2b35-47b2-aa37-096b59c502fd admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:30.261452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cea6f-8ea6-4f31-a469-af9316d0efb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:30.262413 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cea6f-8ea6-4f31-a469-af9316d0efb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:30.262691 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cea6f-8ea6-4f31-a469-af9316d0efb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:30.262975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cea6f-8ea6-4f31-a469-af9316d0efb1 admin admin] PATH_INFO: `/v3/users/1a0eea4729bb403d9fa6cfac657f72fa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:30.265280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5cea6f-8ea6-4f31-a469-af9316d0efb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a0eea4729bb403d9fa6cfac657f72fa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:30.267416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5cea6f-8ea6-4f31-a469-af9316d0efb1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:30.293370 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1073/2167] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:30 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:30.299467 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887088c8-2b35-47b2-aa37-096b59c502fd admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:30.313860 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887088c8-2b35-47b2-aa37-096b59c502fd admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:30.329721 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2198f-2119-49c4-9e75-fa04aecff8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:30.330523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2198f-2119-49c4-9e75-fa04aecff8a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:30.330883 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2198f-2119-49c4-9e75-fa04aecff8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:30.331336 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2198f-2119-49c4-9e75-fa04aecff8a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:30.332301 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2198f-2119-49c4-9e75-fa04aecff8a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:30.334133 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2198f-2119-49c4-9e75-fa04aecff8a1 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:30.356978 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1074/2168] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:30.358007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2198f-2119-49c4-9e75-fa04aecff8a1 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:30.371725 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2198f-2119-49c4-9e75-fa04aecff8a1 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:30.381618 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5d5cea6f-8ea6-4f31-a469-af9316d0efb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a0eea4729bb403d9fa6cfac657f72fa'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:30.388225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32423ec1-9e43-4eba-9632-c6dbabbbe85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:30.391131 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32423ec1-9e43-4eba-9632-c6dbabbbe85d admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:30.391902 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32423ec1-9e43-4eba-9632-c6dbabbbe85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:30.392409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32423ec1-9e43-4eba-9632-c6dbabbbe85d admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:30.395068 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32423ec1-9e43-4eba-9632-c6dbabbbe85d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:30.395743 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32423ec1-9e43-4eba-9632-c6dbabbbe85d admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:30.396988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5d5cea6f-8ea6-4f31-a469-af9316d0efb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a0eea4729bb403d9fa6cfac657f72fa'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:30.413367 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1075/2169] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:30.413888 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5d5cea6f-8ea6-4f31-a469-af9316d0efb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a0eea4729bb403d9fa6cfac657f72fa'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:30.415071 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32423ec1-9e43-4eba-9632-c6dbabbbe85d admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:30.415697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5d5cea6f-8ea6-4f31-a469-af9316d0efb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a0eea4729bb403d9fa6cfac657f72fa'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:30.416693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5d5cea6f-8ea6-4f31-a469-af9316d0efb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a0eea4729bb403d9fa6cfac657f72fa'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:30.417298 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-32423ec1-9e43-4eba-9632-c6dbabbbe85d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:30.442805 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32423ec1-9e43-4eba-9632-c6dbabbbe85d admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:30.453353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c4d01-7b41-4474-9701-55000c080080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:30.454328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c4d01-7b41-4474-9701-55000c080080 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:30.454628 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c4d01-7b41-4474-9701-55000c080080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:30.454923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c4d01-7b41-4474-9701-55000c080080 admin admin] PATH_INFO: `/v3/projects/61ae36d63aa24ed3823f9617c0885266/users/0650740633e6457c9ab2ce8e3b4d877d/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:30.456514 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1095/2170] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:30 2022] DELETE /identity/v3/users/1a0eea4729bb403d9fa6cfac657f72fa => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:30.460819 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93587cf0-1a9c-47e2-84d1-5f71dc956d13 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:30.461112 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93587cf0-1a9c-47e2-84d1-5f71dc956d13 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:30.461398 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93587cf0-1a9c-47e2-84d1-5f71dc956d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:30.474561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:30.482989 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c4d01-7b41-4474-9701-55000c080080 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61ae36d63aa24ed3823f9617c0885266, user_id=0650740633e6457c9ab2ce8e3b4d877d, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:30.492187 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c4d01-7b41-4474-9701-55000c080080 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:30.504456 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-93587cf0-1a9c-47e2-84d1-5f71dc956d13 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:30.506064 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-93587cf0-1a9c-47e2-84d1-5f71dc956d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:30.527661 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1076/2171] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:30 2022] PUT /identity/v3/projects/61ae36d63aa24ed3823f9617c0885266/users/0650740633e6457c9ab2ce8e3b4d877d/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:30.538873 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b67a0d-58c1-4783-b7d3-49768e863431 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:30.539304 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b67a0d-58c1-4783-b7d3-49768e863431 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:30.539624 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b67a0d-58c1-4783-b7d3-49768e863431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:30.587069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1096/2172] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:06:30.587613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-91b67a0d-58c1-4783-b7d3-49768e863431 None None] MFA Rules not processed for user `0650740633e6457c9ab2ce8e3b4d877d`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:30.589160 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-91b67a0d-58c1-4783-b7d3-49768e863431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:30.603636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93587cf0-1a9c-47e2-84d1-5f71dc956d13 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:30.605624 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-93587cf0-1a9c-47e2-84d1-5f71dc956d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:30.688241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1077/2173] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:06:30.692982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b67a0d-58c1-4783-b7d3-49768e863431 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:30.718510 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93587cf0-1a9c-47e2-84d1-5f71dc956d13 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:30.718871 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b67a0d-58c1-4783-b7d3-49768e863431 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:30.722708 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c267bb-9574-4f8b-a92c-fb94307a351d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:30.723395 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c267bb-9574-4f8b-a92c-fb94307a351d service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:30.723684 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c267bb-9574-4f8b-a92c-fb94307a351d service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:30.724141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c267bb-9574-4f8b-a92c-fb94307a351d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:30.734528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98bc784-d7c1-4f86-a9bf-19f1b93c078d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:30.735479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98bc784-d7c1-4f86-a9bf-19f1b93c078d admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:30.735843 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98bc784-d7c1-4f86-a9bf-19f1b93c078d admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:30.736130 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98bc784-d7c1-4f86-a9bf-19f1b93c078d admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:30.736860 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98bc784-d7c1-4f86-a9bf-19f1b93c078d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:30.739847 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98bc784-d7c1-4f86-a9bf-19f1b93c078d admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:30.740074 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c267bb-9574-4f8b-a92c-fb94307a351d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:30.742002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c267bb-9574-4f8b-a92c-fb94307a351d service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:30.780547 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1097/2174] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:30.806415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1078/2175] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:30 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:30.813587 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c267bb-9574-4f8b-a92c-fb94307a351d service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:30.832797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c267bb-9574-4f8b-a92c-fb94307a351d service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:30.943447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b81b9c-08ad-4379-a476-3b5d399ebd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:30.944885 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b81b9c-08ad-4379-a476-3b5d399ebd24 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:30.945268 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b81b9c-08ad-4379-a476-3b5d399ebd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:30.945561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b81b9c-08ad-4379-a476-3b5d399ebd24 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:30.947085 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b81b9c-08ad-4379-a476-3b5d399ebd24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:30.954394 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b81b9c-08ad-4379-a476-3b5d399ebd24 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:31.025757 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1098/2176] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:30 2022] POST /identity/v3/users => generated 299 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:31.031003 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98bc784-d7c1-4f86-a9bf-19f1b93c078d admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:31.043701 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98bc784-d7c1-4f86-a9bf-19f1b93c078d admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:31.055808 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87aafd-543b-40ca-88e1-93c48fb51c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:31.057179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87aafd-543b-40ca-88e1-93c48fb51c7d admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:31.057701 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87aafd-543b-40ca-88e1-93c48fb51c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:31.058484 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87aafd-543b-40ca-88e1-93c48fb51c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:31.065028 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87aafd-543b-40ca-88e1-93c48fb51c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:31.069450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87aafd-543b-40ca-88e1-93c48fb51c7d admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:31.149653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1079/2177] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:31.152377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b81b9c-08ad-4379-a476-3b5d399ebd24 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:31.167944 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b81b9c-08ad-4379-a476-3b5d399ebd24 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:31.199476 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111cf130-74ee-4709-b002-86ee6802f920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:31.209150 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cf130-74ee-4709-b002-86ee6802f920 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:31.210595 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cf130-74ee-4709-b002-86ee6802f920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:31.212424 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cf130-74ee-4709-b002-86ee6802f920 admin admin] PATH_INFO: `/v3/projects/5d1787779ddb4012a6fa02196792ed43/users/7c4d424c131c4ca4bacf9717d115a257/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:31.289281 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111cf130-74ee-4709-b002-86ee6802f920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d1787779ddb4012a6fa02196792ed43, user_id=7c4d424c131c4ca4bacf9717d115a257, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:31.296681 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111cf130-74ee-4709-b002-86ee6802f920 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:31.300484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:31.346091 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1099/2178] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:31 2022] PUT /identity/v3/projects/5d1787779ddb4012a6fa02196792ed43/users/7c4d424c131c4ca4bacf9717d115a257/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:31.347453 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87aafd-543b-40ca-88e1-93c48fb51c7d admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:31.388813 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87aafd-543b-40ca-88e1-93c48fb51c7d admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:31.421740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111cf130-74ee-4709-b002-86ee6802f920 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:31.425196 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-111cf130-74ee-4709-b002-86ee6802f920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:31.594240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:31.668270 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87414e42-dc81-4ad5-871d-ca8013af9b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:31.669009 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87414e42-dc81-4ad5-871d-ca8013af9b80 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:31.669263 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87414e42-dc81-4ad5-871d-ca8013af9b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:31.669545 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87414e42-dc81-4ad5-871d-ca8013af9b80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:31.670380 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87414e42-dc81-4ad5-871d-ca8013af9b80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:31.676750 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87414e42-dc81-4ad5-871d-ca8013af9b80 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:31.706914 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1080/2179] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:31.712769 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87414e42-dc81-4ad5-871d-ca8013af9b80 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:31.730230 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87414e42-dc81-4ad5-871d-ca8013af9b80 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:31.750010 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5582f75-01e7-4b8a-95ff-c64d8b0bdb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:31.750741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5582f75-01e7-4b8a-95ff-c64d8b0bdb3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:31.751040 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5582f75-01e7-4b8a-95ff-c64d8b0bdb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:31.751323 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5582f75-01e7-4b8a-95ff-c64d8b0bdb3f admin admin] PATH_INFO: `/v3/projects/5d1787779ddb4012a6fa02196792ed43/users/7c4d424c131c4ca4bacf9717d115a257/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:31.770032 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5582f75-01e7-4b8a-95ff-c64d8b0bdb3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d1787779ddb4012a6fa02196792ed43, user_id=7c4d424c131c4ca4bacf9717d115a257, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:31.777686 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111cf130-74ee-4709-b002-86ee6802f920 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:31.782816 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5582f75-01e7-4b8a-95ff-c64d8b0bdb3f admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:31.805163 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1081/2180] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:31 2022] PUT /identity/v3/projects/5d1787779ddb4012a6fa02196792ed43/users/7c4d424c131c4ca4bacf9717d115a257/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:31.828986 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f1761-9807-457b-b71e-104041519b28 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:31.829725 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f1761-9807-457b-b71e-104041519b28 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:31.830173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f1761-9807-457b-b71e-104041519b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:31.832009 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb24b3-2788-4845-802e-b1186ffcd532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:31.833588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb24b3-2788-4845-802e-b1186ffcd532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:31.834075 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb24b3-2788-4845-802e-b1186ffcd532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:31.841584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb24b3-2788-4845-802e-b1186ffcd532 admin admin] PATH_INFO: `/v3/projects/1f2297508a2a4ef0a8cfceedc4fad02f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:31.850666 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb24b3-2788-4845-802e-b1186ffcd532 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f2297508a2a4ef0a8cfceedc4fad02f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:31.862979 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb24b3-2788-4845-802e-b1186ffcd532 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:31.915445 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-9a5f1761-9807-457b-b71e-104041519b28 None None] MFA Rules not processed for user `7c4d424c131c4ca4bacf9717d115a257`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:31.920733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-9a5f1761-9807-457b-b71e-104041519b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:32.052645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d0bb24b3-2788-4845-802e-b1186ffcd532 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f2297508a2a4ef0a8cfceedc4fad02f', 'request_id': 'req-d0bb24b3-2788-4845-802e-b1186ffcd532'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:32.070966 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d0bb24b3-2788-4845-802e-b1186ffcd532 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f2297508a2a4ef0a8cfceedc4fad02f', 'request_id': 'req-d0bb24b3-2788-4845-802e-b1186ffcd532'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:32.107646 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d0bb24b3-2788-4845-802e-b1186ffcd532 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f2297508a2a4ef0a8cfceedc4fad02f', 'request_id': 'req-d0bb24b3-2788-4845-802e-b1186ffcd532'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:32.163009 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d0bb24b3-2788-4845-802e-b1186ffcd532 admin admin] The token cache is being invalidate because project 1f2297508a2a4ef0a8cfceedc4fad02f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:32.163311 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d0bb24b3-2788-4845-802e-b1186ffcd532 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:32.164432 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d0bb24b3-2788-4845-802e-b1186ffcd532 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:32.173040 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1100/2181] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:31 2022] DELETE /identity/v3/projects/1f2297508a2a4ef0a8cfceedc4fad02f => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:32.177491 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb24b3-2788-4845-802e-b1186ffcd532 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:32.179786 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d0bb24b3-2788-4845-802e-b1186ffcd532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:32.199098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1082/2182] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:32.200838 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d7160-c1e4-48f9-9392-2ad5812955b8 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:32.201163 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d7160-c1e4-48f9-9392-2ad5812955b8 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:32.203342 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d7160-c1e4-48f9-9392-2ad5812955b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:32.239703 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-354d7160-c1e4-48f9-9392-2ad5812955b8 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:32.242517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-354d7160-c1e4-48f9-9392-2ad5812955b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:32.334657 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb24b3-2788-4845-802e-b1186ffcd532 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:32.345447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce7bb4-ee88-4f6a-9e53-0f03e1720a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:32.346320 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce7bb4-ee88-4f6a-9e53-0f03e1720a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:32.346695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce7bb4-ee88-4f6a-9e53-0f03e1720a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:32.347070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce7bb4-ee88-4f6a-9e53-0f03e1720a50 admin admin] PATH_INFO: `/v3/users/d39a402d0e7b4220b787d9c8ff30a1cb` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:32.349806 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce7bb4-ee88-4f6a-9e53-0f03e1720a50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d39a402d0e7b4220b787d9c8ff30a1cb)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:32.352632 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce7bb4-ee88-4f6a-9e53-0f03e1720a50 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:32.366500 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1083/2183] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:32.383385 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d7160-c1e4-48f9-9392-2ad5812955b8 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:32.383385 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-354d7160-c1e4-48f9-9392-2ad5812955b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:32.407896 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:INSERT=2,keystone:SELECT=87,keystone:DELETE=11 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:32.438431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-66ce7bb4-ee88-4f6a-9e53-0f03e1720a50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd39a402d0e7b4220b787d9c8ff30a1cb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:32.446558 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-66ce7bb4-ee88-4f6a-9e53-0f03e1720a50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd39a402d0e7b4220b787d9c8ff30a1cb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:32.448535 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d7160-c1e4-48f9-9392-2ad5812955b8 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:32.452325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f7623-f06c-46e2-9183-5df2e41efdc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:32.452993 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f7623-f06c-46e2-9183-5df2e41efdc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:32.453284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f7623-f06c-46e2-9183-5df2e41efdc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:32.453556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f7623-f06c-46e2-9183-5df2e41efdc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:32.455442 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-861f7623-f06c-46e2-9183-5df2e41efdc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:32.461095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-66ce7bb4-ee88-4f6a-9e53-0f03e1720a50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd39a402d0e7b4220b787d9c8ff30a1cb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:32.462165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-66ce7bb4-ee88-4f6a-9e53-0f03e1720a50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd39a402d0e7b4220b787d9c8ff30a1cb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:32.463418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-66ce7bb4-ee88-4f6a-9e53-0f03e1720a50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd39a402d0e7b4220b787d9c8ff30a1cb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:32.468888 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:32.480828 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1101/2184] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:32 2022] DELETE /identity/v3/users/d39a402d0e7b4220b787d9c8ff30a1cb => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:32.482392 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a29f1f-0c54-4a06-a52c-43cf19a39526 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:32.482776 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a29f1f-0c54-4a06-a52c-43cf19a39526 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:32.483985 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a29f1f-0c54-4a06-a52c-43cf19a39526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:32.507612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e8a29f1f-0c54-4a06-a52c-43cf19a39526 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:32.509015 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e8a29f1f-0c54-4a06-a52c-43cf19a39526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:32.546553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f7623-f06c-46e2-9183-5df2e41efdc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:32.548122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f7623-f06c-46e2-9183-5df2e41efdc6 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:32.559561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1084/2185] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:32.560505 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49036c69-3731-4036-a3b3-371d25086097 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:32.560797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49036c69-3731-4036-a3b3-371d25086097 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:32.561906 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49036c69-3731-4036-a3b3-371d25086097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:32.579960 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1102/2186] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:32.581484 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-49036c69-3731-4036-a3b3-371d25086097 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:32.582749 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-49036c69-3731-4036-a3b3-371d25086097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:32.584550 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a29f1f-0c54-4a06-a52c-43cf19a39526 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:32.592937 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1085/2187] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:32.595928 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a29f1f-0c54-4a06-a52c-43cf19a39526 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:32.602299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad417f1-ec86-4b77-bf9c-c2b1d8166a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:32.604016 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad417f1-ec86-4b77-bf9c-c2b1d8166a9b admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:32.604298 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad417f1-ec86-4b77-bf9c-c2b1d8166a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:32.604604 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad417f1-ec86-4b77-bf9c-c2b1d8166a9b admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:32.605371 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad417f1-ec86-4b77-bf9c-c2b1d8166a9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:06:32.605681 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad417f1-ec86-4b77-bf9c-c2b1d8166a9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:32.607434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad417f1-ec86-4b77-bf9c-c2b1d8166a9b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:32.609308 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49036c69-3731-4036-a3b3-371d25086097 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:32.611595 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-49036c69-3731-4036-a3b3-371d25086097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:32.623787 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1103/2188] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:06:32.631465 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad417f1-ec86-4b77-bf9c-c2b1d8166a9b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:32.645326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad417f1-ec86-4b77-bf9c-c2b1d8166a9b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:32.651378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d0fba-b292-4148-8625-7ff3eca3c8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:32.652186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d0fba-b292-4148-8625-7ff3eca3c8f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:32.652607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d0fba-b292-4148-8625-7ff3eca3c8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:32.652885 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d0fba-b292-4148-8625-7ff3eca3c8f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:32.653586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d0fba-b292-4148-8625-7ff3eca3c8f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:32.657762 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d0fba-b292-4148-8625-7ff3eca3c8f3 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:32.695459 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1104/2189] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:32 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:32.699763 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d0fba-b292-4148-8625-7ff3eca3c8f3 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:32.700772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49036c69-3731-4036-a3b3-371d25086097 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:32.792164 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187464a8-17ea-4e25-ba56-2772fe27294d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:32.792843 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187464a8-17ea-4e25-ba56-2772fe27294d service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:32.793132 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187464a8-17ea-4e25-ba56-2772fe27294d service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:32.793413 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187464a8-17ea-4e25-ba56-2772fe27294d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:32.837777 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d0fba-b292-4148-8625-7ff3eca3c8f3 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:32.849139 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2edbcc-1718-4e60-a470-77c855589d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:32.849139 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2edbcc-1718-4e60-a470-77c855589d86 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:32.849139 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2edbcc-1718-4e60-a470-77c855589d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:32.849139 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2edbcc-1718-4e60-a470-77c855589d86 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:32.849606 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2edbcc-1718-4e60-a470-77c855589d86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:32.851841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2edbcc-1718-4e60-a470-77c855589d86 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:32.874467 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:32.889493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187464a8-17ea-4e25-ba56-2772fe27294d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:32.891514 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187464a8-17ea-4e25-ba56-2772fe27294d service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:32.892501 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1105/2190] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:32 2022] POST /identity/v3/users => generated 288 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:06:32.893556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2edbcc-1718-4e60-a470-77c855589d86 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:32.909479 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2edbcc-1718-4e60-a470-77c855589d86 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:32.910395 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1086/2191] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:32.940778 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53823a63-5f12-441f-9b0c-ab325f1998d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:32.941913 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53823a63-5f12-441f-9b0c-ab325f1998d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:32.942434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53823a63-5f12-441f-9b0c-ab325f1998d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:32.942912 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53823a63-5f12-441f-9b0c-ab325f1998d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:32.945382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53823a63-5f12-441f-9b0c-ab325f1998d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:32.948826 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53823a63-5f12-441f-9b0c-ab325f1998d7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:32.964334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1106/2192] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:32.969044 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187464a8-17ea-4e25-ba56-2772fe27294d service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:32.979912 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187464a8-17ea-4e25-ba56-2772fe27294d service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:32.985745 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa7126-0671-4847-9a8f-7f0efbfbb437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:32.986829 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa7126-0671-4847-9a8f-7f0efbfbb437 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:32.987188 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa7126-0671-4847-9a8f-7f0efbfbb437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:32.987533 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa7126-0671-4847-9a8f-7f0efbfbb437 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:32.988416 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa7126-0671-4847-9a8f-7f0efbfbb437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:32.990453 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa7126-0671-4847-9a8f-7f0efbfbb437 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:32.992110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:33.006519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1087/2193] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:06:33.011531 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53823a63-5f12-441f-9b0c-ab325f1998d7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:33.040370 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53823a63-5f12-441f-9b0c-ab325f1998d7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:33.047457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ed099-2ce7-4138-9c2a-34c06b763eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:33.048669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ed099-2ce7-4138-9c2a-34c06b763eb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:33.049139 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ed099-2ce7-4138-9c2a-34c06b763eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:33.049606 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ed099-2ce7-4138-9c2a-34c06b763eb5 admin admin] PATH_INFO: `/v3/projects/71fe7e98fefc4471be3cb40564a67411/users/c3e1ac754d9843a881835442cade5d1e/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:33.074161 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ed099-2ce7-4138-9c2a-34c06b763eb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71fe7e98fefc4471be3cb40564a67411, user_id=c3e1ac754d9843a881835442cade5d1e, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:33.079984 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ed099-2ce7-4138-9c2a-34c06b763eb5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:33.110655 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1107/2194] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:33 2022] PUT /identity/v3/projects/71fe7e98fefc4471be3cb40564a67411/users/c3e1ac754d9843a881835442cade5d1e/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:06:33.115437 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033362ac-6e15-47e4-b6a4-5fc6697ad2e4 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:33.115733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033362ac-6e15-47e4-b6a4-5fc6697ad2e4 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:33.116102 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033362ac-6e15-47e4-b6a4-5fc6697ad2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:33.170907 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-033362ac-6e15-47e4-b6a4-5fc6697ad2e4 None None] MFA Rules not processed for user `c3e1ac754d9843a881835442cade5d1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:33.172453 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-033362ac-6e15-47e4-b6a4-5fc6697ad2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:33.282710 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1088/2195] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:06:33.295528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ed099-2ce7-4138-9c2a-34c06b763eb5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:33.299030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-760ed099-2ce7-4138-9c2a-34c06b763eb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:33.304004 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033362ac-6e15-47e4-b6a4-5fc6697ad2e4 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:33.314652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-033362ac-6e15-47e4-b6a4-5fc6697ad2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:33.456574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ed099-2ce7-4138-9c2a-34c06b763eb5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:33.469945 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd47d8-48a4-4bb0-9782-2f1f8a5462be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:33.471332 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd47d8-48a4-4bb0-9782-2f1f8a5462be service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:33.472413 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd47d8-48a4-4bb0-9782-2f1f8a5462be service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:33.472874 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd47d8-48a4-4bb0-9782-2f1f8a5462be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:33.487137 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033362ac-6e15-47e4-b6a4-5fc6697ad2e4 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:33.490652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdd47d8-48a4-4bb0-9782-2f1f8a5462be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:33.492547 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdd47d8-48a4-4bb0-9782-2f1f8a5462be service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:33.501103 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77015e95-ac8e-476b-908b-c4e7233c5a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:33.503544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77015e95-ac8e-476b-908b-c4e7233c5a24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:33.503544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77015e95-ac8e-476b-908b-c4e7233c5a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:33.503544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77015e95-ac8e-476b-908b-c4e7233c5a24 admin admin] PATH_INFO: `/v3/projects/f1056091fba74e56a14f77d588f33afc` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:33.506556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77015e95-ac8e-476b-908b-c4e7233c5a24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1056091fba74e56a14f77d588f33afc)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:33.508534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77015e95-ac8e-476b-908b-c4e7233c5a24 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:33.513838 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1108/2196] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:33.526370 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd47d8-48a4-4bb0-9782-2f1f8a5462be service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:33.528196 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-afdd47d8-48a4-4bb0-9782-2f1f8a5462be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:33.556108 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd47d8-48a4-4bb0-9782-2f1f8a5462be service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:33.566248 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f89ff-8a88-41d3-b41e-641e3de1b5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:33.567543 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f89ff-8a88-41d3-b41e-641e3de1b5f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:33.568205 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f89ff-8a88-41d3-b41e-641e3de1b5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:33.568687 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f89ff-8a88-41d3-b41e-641e3de1b5f0 admin admin] PATH_INFO: `/v3/users/0650740633e6457c9ab2ce8e3b4d877d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:33.571151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f89ff-8a88-41d3-b41e-641e3de1b5f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0650740633e6457c9ab2ce8e3b4d877d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:33.573643 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f89ff-8a88-41d3-b41e-641e3de1b5f0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:33.585859 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-77015e95-ac8e-476b-908b-c4e7233c5a24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1056091fba74e56a14f77d588f33afc', 'request_id': 'req-77015e95-ac8e-476b-908b-c4e7233c5a24'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:33.596226 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-77015e95-ac8e-476b-908b-c4e7233c5a24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1056091fba74e56a14f77d588f33afc', 'request_id': 'req-77015e95-ac8e-476b-908b-c4e7233c5a24'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:33.603316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-77015e95-ac8e-476b-908b-c4e7233c5a24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1056091fba74e56a14f77d588f33afc', 'request_id': 'req-77015e95-ac8e-476b-908b-c4e7233c5a24'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:33.618830 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-77015e95-ac8e-476b-908b-c4e7233c5a24 admin admin] The token cache is being invalidate because project f1056091fba74e56a14f77d588f33afc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:33.619126 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-77015e95-ac8e-476b-908b-c4e7233c5a24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:33.620507 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-77015e95-ac8e-476b-908b-c4e7233c5a24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:33.637126 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1089/2197] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:33 2022] DELETE /identity/v3/projects/f1056091fba74e56a14f77d588f33afc => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:33.645593 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77015e95-ac8e-476b-908b-c4e7233c5a24 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:33.647171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-77015e95-ac8e-476b-908b-c4e7233c5a24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:33.647554 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-229f89ff-8a88-41d3-b41e-641e3de1b5f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0650740633e6457c9ab2ce8e3b4d877d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:33.662543 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-229f89ff-8a88-41d3-b41e-641e3de1b5f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0650740633e6457c9ab2ce8e3b4d877d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:33.674270 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:33.678925 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-229f89ff-8a88-41d3-b41e-641e3de1b5f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0650740633e6457c9ab2ce8e3b4d877d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:33.679941 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-229f89ff-8a88-41d3-b41e-641e3de1b5f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0650740633e6457c9ab2ce8e3b4d877d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:33.680673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-229f89ff-8a88-41d3-b41e-641e3de1b5f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0650740633e6457c9ab2ce8e3b4d877d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:33.699712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1109/2198] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:33 2022] DELETE /identity/v3/users/0650740633e6457c9ab2ce8e3b4d877d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:33.702526 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22df334-8f7a-4209-a4af-accfd24f7c23 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:33.702823 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22df334-8f7a-4209-a4af-accfd24f7c23 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:33.703105 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22df334-8f7a-4209-a4af-accfd24f7c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:33.724627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e22df334-8f7a-4209-a4af-accfd24f7c23 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:33.726706 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e22df334-8f7a-4209-a4af-accfd24f7c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:33.736519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77015e95-ac8e-476b-908b-c4e7233c5a24 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:33.776310 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:33.814350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9db527-31a4-41c6-bbbf-0a1ae77c8694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:33.815534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9db527-31a4-41c6-bbbf-0a1ae77c8694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:33.815828 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9db527-31a4-41c6-bbbf-0a1ae77c8694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:33.816108 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9db527-31a4-41c6-bbbf-0a1ae77c8694 admin admin] PATH_INFO: `/v3/users/ff6b4dd95538450e8ff180122a0be531` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:33.817446 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9db527-31a4-41c6-bbbf-0a1ae77c8694 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff6b4dd95538450e8ff180122a0be531)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:33.819714 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9db527-31a4-41c6-bbbf-0a1ae77c8694 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:33.845657 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1110/2199] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:33.858293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22df334-8f7a-4209-a4af-accfd24f7c23 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:33.863753 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e22df334-8f7a-4209-a4af-accfd24f7c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:33.921832 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-4f9db527-31a4-41c6-bbbf-0a1ae77c8694 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff6b4dd95538450e8ff180122a0be531'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:33.933477 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-4f9db527-31a4-41c6-bbbf-0a1ae77c8694 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff6b4dd95538450e8ff180122a0be531'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:33.934240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-4f9db527-31a4-41c6-bbbf-0a1ae77c8694 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff6b4dd95538450e8ff180122a0be531'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:33.935365 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-4f9db527-31a4-41c6-bbbf-0a1ae77c8694 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff6b4dd95538450e8ff180122a0be531'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:33.948816 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-4f9db527-31a4-41c6-bbbf-0a1ae77c8694 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff6b4dd95538450e8ff180122a0be531'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:33.956166 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22df334-8f7a-4209-a4af-accfd24f7c23 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:33.961853 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6580170-30bd-4c8d-a31f-6dcce7c2533f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:33.962747 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6580170-30bd-4c8d-a31f-6dcce7c2533f service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:33.963014 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6580170-30bd-4c8d-a31f-6dcce7c2533f service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:33.963306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6580170-30bd-4c8d-a31f-6dcce7c2533f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:33.965551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c6580170-30bd-4c8d-a31f-6dcce7c2533f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:33.982592 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1090/2200] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:33 2022] DELETE /identity/v3/users/ff6b4dd95538450e8ff180122a0be531 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:33.986565 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6580170-30bd-4c8d-a31f-6dcce7c2533f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:33.988048 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6580170-30bd-4c8d-a31f-6dcce7c2533f service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:34.000230 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1111/2201] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:34.403972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9db527-31a4-41c6-bbbf-0a1ae77c8694 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:34.405961 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-4f9db527-31a4-41c6-bbbf-0a1ae77c8694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:34.434908 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6580170-30bd-4c8d-a31f-6dcce7c2533f service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:34.438253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c6580170-30bd-4c8d-a31f-6dcce7c2533f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:34.463655 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6580170-30bd-4c8d-a31f-6dcce7c2533f service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:34.474244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:34.497781 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9db527-31a4-41c6-bbbf-0a1ae77c8694 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:34.503393 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba0e31-3800-4737-9e03-d8b22fe0c560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:34.504308 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba0e31-3800-4737-9e03-d8b22fe0c560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:34.504654 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba0e31-3800-4737-9e03-d8b22fe0c560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:34.504939 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba0e31-3800-4737-9e03-d8b22fe0c560 admin admin] PATH_INFO: `/v3/projects/5bf803da3c0343ed831c6ded0df4fba8` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:34.506937 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba0e31-3800-4737-9e03-d8b22fe0c560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bf803da3c0343ed831c6ded0df4fba8)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:34.508981 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba0e31-3800-4737-9e03-d8b22fe0c560 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:34.539618 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a37c1-7df0-426e-997e-caaa409cc927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:34.540475 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a37c1-7df0-426e-997e-caaa409cc927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:34.540790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a37c1-7df0-426e-997e-caaa409cc927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:34.541093 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a37c1-7df0-426e-997e-caaa409cc927 admin admin] PATH_INFO: `/v3/projects/61ae36d63aa24ed3823f9617c0885266` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:34.542706 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a37c1-7df0-426e-997e-caaa409cc927 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61ae36d63aa24ed3823f9617c0885266)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:34.544848 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a37c1-7df0-426e-997e-caaa409cc927 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:34.568874 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-98ba0e31-3800-4737-9e03-d8b22fe0c560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bf803da3c0343ed831c6ded0df4fba8', 'request_id': 'req-98ba0e31-3800-4737-9e03-d8b22fe0c560'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:34.584091 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-98ba0e31-3800-4737-9e03-d8b22fe0c560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bf803da3c0343ed831c6ded0df4fba8', 'request_id': 'req-98ba0e31-3800-4737-9e03-d8b22fe0c560'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:34.591858 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-98ba0e31-3800-4737-9e03-d8b22fe0c560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bf803da3c0343ed831c6ded0df4fba8', 'request_id': 'req-98ba0e31-3800-4737-9e03-d8b22fe0c560'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:34.607389 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-98ba0e31-3800-4737-9e03-d8b22fe0c560 admin admin] The token cache is being invalidate because project 5bf803da3c0343ed831c6ded0df4fba8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:34.607771 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-98ba0e31-3800-4737-9e03-d8b22fe0c560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:34.608551 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-98ba0e31-3800-4737-9e03-d8b22fe0c560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:34.610305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1091/2202] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:34 2022] DELETE /identity/v3/projects/5bf803da3c0343ed831c6ded0df4fba8 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:34.614599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba0e31-3800-4737-9e03-d8b22fe0c560 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:34.615978 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-98ba0e31-3800-4737-9e03-d8b22fe0c560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:34.619583 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6a0a37c1-7df0-426e-997e-caaa409cc927 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61ae36d63aa24ed3823f9617c0885266', 'request_id': 'req-6a0a37c1-7df0-426e-997e-caaa409cc927'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:34.626021 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6a0a37c1-7df0-426e-997e-caaa409cc927 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61ae36d63aa24ed3823f9617c0885266', 'request_id': 'req-6a0a37c1-7df0-426e-997e-caaa409cc927'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:34.634389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6a0a37c1-7df0-426e-997e-caaa409cc927 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61ae36d63aa24ed3823f9617c0885266', 'request_id': 'req-6a0a37c1-7df0-426e-997e-caaa409cc927'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:34.649093 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6a0a37c1-7df0-426e-997e-caaa409cc927 admin admin] The token cache is being invalidate because project 61ae36d63aa24ed3823f9617c0885266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:34.649440 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6a0a37c1-7df0-426e-997e-caaa409cc927 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:34.650048 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6a0a37c1-7df0-426e-997e-caaa409cc927 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:34.651640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1112/2203] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:34 2022] DELETE /identity/v3/projects/61ae36d63aa24ed3823f9617c0885266 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:34.656482 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a37c1-7df0-426e-997e-caaa409cc927 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:34.659281 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6a0a37c1-7df0-426e-997e-caaa409cc927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:34.674357 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:34.705728 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba0e31-3800-4737-9e03-d8b22fe0c560 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:34.754665 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a37c1-7df0-426e-997e-caaa409cc927 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:34.764938 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f1800-34a0-4c1f-b047-94ae4cd1a8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:34.766998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f1800-34a0-4c1f-b047-94ae4cd1a8b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:34.767520 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f1800-34a0-4c1f-b047-94ae4cd1a8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:34.768274 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f1800-34a0-4c1f-b047-94ae4cd1a8b8 admin admin] PATH_INFO: `/v3/users/7c4d424c131c4ca4bacf9717d115a257` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:34.770054 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f1800-34a0-4c1f-b047-94ae4cd1a8b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c4d424c131c4ca4bacf9717d115a257)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:34.777394 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f1800-34a0-4c1f-b047-94ae4cd1a8b8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:34.793401 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f51bb-f717-467d-8ce9-2372d255319d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:34.794396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f51bb-f717-467d-8ce9-2372d255319d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:34.794754 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f51bb-f717-467d-8ce9-2372d255319d admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:34.795135 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f51bb-f717-467d-8ce9-2372d255319d admin admin] PATH_INFO: `/v3/users/f9781166ed354aa2b36dfa2e90c53617` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:34.798042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f51bb-f717-467d-8ce9-2372d255319d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9781166ed354aa2b36dfa2e90c53617)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:34.800828 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f51bb-f717-467d-8ce9-2372d255319d admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:34.834880 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:34.858425 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f43f1800-34a0-4c1f-b047-94ae4cd1a8b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c4d424c131c4ca4bacf9717d115a257'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:34.869341 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f43f1800-34a0-4c1f-b047-94ae4cd1a8b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c4d424c131c4ca4bacf9717d115a257'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:34.873438 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-636f51bb-f717-467d-8ce9-2372d255319d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9781166ed354aa2b36dfa2e90c53617'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:34.883978 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-636f51bb-f717-467d-8ce9-2372d255319d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9781166ed354aa2b36dfa2e90c53617'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:34.884613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-636f51bb-f717-467d-8ce9-2372d255319d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9781166ed354aa2b36dfa2e90c53617'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:34.885308 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-636f51bb-f717-467d-8ce9-2372d255319d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9781166ed354aa2b36dfa2e90c53617'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:34.890393 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f43f1800-34a0-4c1f-b047-94ae4cd1a8b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c4d424c131c4ca4bacf9717d115a257'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:34.891454 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f43f1800-34a0-4c1f-b047-94ae4cd1a8b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c4d424c131c4ca4bacf9717d115a257'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:34.892444 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f43f1800-34a0-4c1f-b047-94ae4cd1a8b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c4d424c131c4ca4bacf9717d115a257'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:34.901415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-636f51bb-f717-467d-8ce9-2372d255319d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9781166ed354aa2b36dfa2e90c53617'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:34.915220 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1113/2204] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:34 2022] DELETE /identity/v3/users/7c4d424c131c4ca4bacf9717d115a257 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:06:34.933816 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1092/2205] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:34 2022] DELETE /identity/v3/users/f9781166ed354aa2b36dfa2e90c53617 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:06:35.340113 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f1800-34a0-4c1f-b047-94ae4cd1a8b8 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:35.342307 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f43f1800-34a0-4c1f-b047-94ae4cd1a8b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:35.367688 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f51bb-f717-467d-8ce9-2372d255319d admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:35.369680 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-636f51bb-f717-467d-8ce9-2372d255319d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:35.484175 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f1800-34a0-4c1f-b047-94ae4cd1a8b8 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:35.485865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f51bb-f717-467d-8ce9-2372d255319d admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:35.491129 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3762b08-dcfb-4f78-a4e5-d2d99dbc7750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:35.492348 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3762b08-dcfb-4f78-a4e5-d2d99dbc7750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:35.492632 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3762b08-dcfb-4f78-a4e5-d2d99dbc7750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:35.492961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3762b08-dcfb-4f78-a4e5-d2d99dbc7750 admin admin] PATH_INFO: `/v3/projects/5d1787779ddb4012a6fa02196792ed43` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:35.494021 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df60331f-0a21-4cef-b8ac-ad1a819e8362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:35.494409 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3762b08-dcfb-4f78-a4e5-d2d99dbc7750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d1787779ddb4012a6fa02196792ed43)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:35.495293 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60331f-0a21-4cef-b8ac-ad1a819e8362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:35.495732 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60331f-0a21-4cef-b8ac-ad1a819e8362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:35.496196 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60331f-0a21-4cef-b8ac-ad1a819e8362 admin admin] PATH_INFO: `/v3/projects/90c0d211518e4254acebf59e345921ea` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:35.496667 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3762b08-dcfb-4f78-a4e5-d2d99dbc7750 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:35.497868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df60331f-0a21-4cef-b8ac-ad1a819e8362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90c0d211518e4254acebf59e345921ea)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:35.500085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df60331f-0a21-4cef-b8ac-ad1a819e8362 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:35.549301 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:35.566058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-df60331f-0a21-4cef-b8ac-ad1a819e8362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90c0d211518e4254acebf59e345921ea', 'request_id': 'req-df60331f-0a21-4cef-b8ac-ad1a819e8362'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:35.571391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e3762b08-dcfb-4f78-a4e5-d2d99dbc7750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d1787779ddb4012a6fa02196792ed43', 'request_id': 'req-e3762b08-dcfb-4f78-a4e5-d2d99dbc7750'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:35.575873 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-df60331f-0a21-4cef-b8ac-ad1a819e8362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90c0d211518e4254acebf59e345921ea', 'request_id': 'req-df60331f-0a21-4cef-b8ac-ad1a819e8362'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:35.580668 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e3762b08-dcfb-4f78-a4e5-d2d99dbc7750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d1787779ddb4012a6fa02196792ed43', 'request_id': 'req-e3762b08-dcfb-4f78-a4e5-d2d99dbc7750'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:35.585162 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-df60331f-0a21-4cef-b8ac-ad1a819e8362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90c0d211518e4254acebf59e345921ea', 'request_id': 'req-df60331f-0a21-4cef-b8ac-ad1a819e8362'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:35.603906 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e3762b08-dcfb-4f78-a4e5-d2d99dbc7750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d1787779ddb4012a6fa02196792ed43', 'request_id': 'req-e3762b08-dcfb-4f78-a4e5-d2d99dbc7750'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:35.619608 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-df60331f-0a21-4cef-b8ac-ad1a819e8362 admin admin] The token cache is being invalidate because project 90c0d211518e4254acebf59e345921ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:35.619608 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-df60331f-0a21-4cef-b8ac-ad1a819e8362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:35.620625 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-df60331f-0a21-4cef-b8ac-ad1a819e8362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:35.622675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1093/2206] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:35 2022] DELETE /identity/v3/projects/90c0d211518e4254acebf59e345921ea => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:06:35.631059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e3762b08-dcfb-4f78-a4e5-d2d99dbc7750 admin admin] The token cache is being invalidate because project 5d1787779ddb4012a6fa02196792ed43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:35.631428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e3762b08-dcfb-4f78-a4e5-d2d99dbc7750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:35.631602 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d365c2c-f8eb-4956-a24c-858c7c8c5296 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:35.632043 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e3762b08-dcfb-4f78-a4e5-d2d99dbc7750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:35.634174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1114/2207] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:35 2022] DELETE /identity/v3/projects/5d1787779ddb4012a6fa02196792ed43 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:35.634343 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d365c2c-f8eb-4956-a24c-858c7c8c5296 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:35.634675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d365c2c-f8eb-4956-a24c-858c7c8c5296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:35.640028 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442ac70-e452-48c1-b567-57271efee96a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:35.640470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442ac70-e452-48c1-b567-57271efee96a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:35.641083 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442ac70-e452-48c1-b567-57271efee96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:35.662460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5442ac70-e452-48c1-b567-57271efee96a None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:35.664365 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5442ac70-e452-48c1-b567-57271efee96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:35.672483 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-0d365c2c-f8eb-4956-a24c-858c7c8c5296 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:35.674358 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-0d365c2c-f8eb-4956-a24c-858c7c8c5296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:35.715828 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:35.745605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1115/2208] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:35.755097 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093cc9e-c761-4427-b2a7-14796252693f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:35.755407 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093cc9e-c761-4427-b2a7-14796252693f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:35.755702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093cc9e-c761-4427-b2a7-14796252693f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:35.762740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1094/2209] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:35.772736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2056c-7e68-421b-8f6b-1b5a33d47097 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:35.773141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2056c-7e68-421b-8f6b-1b5a33d47097 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:35.773433 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2056c-7e68-421b-8f6b-1b5a33d47097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:35.776621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2093cc9e-c761-4427-b2a7-14796252693f None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:35.777856 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2093cc9e-c761-4427-b2a7-14796252693f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:35.786669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1116/2210] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:35.790852 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093cc9e-c761-4427-b2a7-14796252693f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:35.795761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-4ac2056c-7e68-421b-8f6b-1b5a33d47097 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:35.797186 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-4ac2056c-7e68-421b-8f6b-1b5a33d47097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:35.801187 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093cc9e-c761-4427-b2a7-14796252693f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:35.806392 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccd797-887b-4f58-ad1b-6175de820fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:35.806557 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1095/2211] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:06:35.807381 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccd797-887b-4f58-ad1b-6175de820fdc admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:35.807609 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccd797-887b-4f58-ad1b-6175de820fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:35.807886 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccd797-887b-4f58-ad1b-6175de820fdc admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:35.808614 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ccd797-887b-4f58-ad1b-6175de820fdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:06:35.808922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ccd797-887b-4f58-ad1b-6175de820fdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:35.810394 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2056c-7e68-421b-8f6b-1b5a33d47097 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:35.810683 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ccd797-887b-4f58-ad1b-6175de820fdc admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:35.821100 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2056c-7e68-421b-8f6b-1b5a33d47097 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:35.826541 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa48e3-5c6c-4ff2-8b0c-3c0c5f62f8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:35.827916 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa48e3-5c6c-4ff2-8b0c-3c0c5f62f8f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:35.828299 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa48e3-5c6c-4ff2-8b0c-3c0c5f62f8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:35.828609 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1117/2212] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:06:35.828720 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa48e3-5c6c-4ff2-8b0c-3c0c5f62f8f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:35.829482 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa48e3-5c6c-4ff2-8b0c-3c0c5f62f8f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:06:35.829864 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa48e3-5c6c-4ff2-8b0c-3c0c5f62f8f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:35.831655 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa48e3-5c6c-4ff2-8b0c-3c0c5f62f8f3 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:35.832999 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccd797-887b-4f58-ad1b-6175de820fdc admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:35.846231 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1096/2213] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:06:35.849354 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccd797-887b-4f58-ad1b-6175de820fdc admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:35.851363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa48e3-5c6c-4ff2-8b0c-3c0c5f62f8f3 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:35.856312 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e526b-fc6b-4b90-b89e-aca8e5ca0671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:35.857335 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e526b-fc6b-4b90-b89e-aca8e5ca0671 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:35.857703 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e526b-fc6b-4b90-b89e-aca8e5ca0671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:35.860635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e526b-fc6b-4b90-b89e-aca8e5ca0671 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:35.861467 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e526b-fc6b-4b90-b89e-aca8e5ca0671 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:35.862474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa48e3-5c6c-4ff2-8b0c-3c0c5f62f8f3 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:35.863862 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e526b-fc6b-4b90-b89e-aca8e5ca0671 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:35.867515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce076d1-4e2a-4041-8300-04ca3eacd79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:35.868184 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce076d1-4e2a-4041-8300-04ca3eacd79e admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:35.868472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce076d1-4e2a-4041-8300-04ca3eacd79e admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:35.868913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce076d1-4e2a-4041-8300-04ca3eacd79e admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:35.869453 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce076d1-4e2a-4041-8300-04ca3eacd79e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:35.871497 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce076d1-4e2a-4041-8300-04ca3eacd79e admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:35.900877 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1118/2214] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:35 2022] POST /identity/v3/projects => generated 369 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:06:35.901688 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e526b-fc6b-4b90-b89e-aca8e5ca0671 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:35.918800 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1097/2215] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:35 2022] POST /identity/v3/projects => generated 401 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:35.922906 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce076d1-4e2a-4041-8300-04ca3eacd79e admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:36.002768 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e526b-fc6b-4b90-b89e-aca8e5ca0671 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:36.048906 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:36.067056 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce076d1-4e2a-4041-8300-04ca3eacd79e admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:36.073469 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0740243-15e5-46a9-93e3-a6e5d5be0dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:36.074459 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0740243-15e5-46a9-93e3-a6e5d5be0dc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:36.074750 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0740243-15e5-46a9-93e3-a6e5d5be0dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:36.075457 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0740243-15e5-46a9-93e3-a6e5d5be0dc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:36.076341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0740243-15e5-46a9-93e3-a6e5d5be0dc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:36.078423 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0740243-15e5-46a9-93e3-a6e5d5be0dc5 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:36.100812 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09733062-a1d8-45af-8894-447282cc572a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:36.101761 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09733062-a1d8-45af-8894-447282cc572a admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:36.102088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09733062-a1d8-45af-8894-447282cc572a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:36.102437 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09733062-a1d8-45af-8894-447282cc572a admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:36.103361 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09733062-a1d8-45af-8894-447282cc572a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:36.105604 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09733062-a1d8-45af-8894-447282cc572a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:36.108568 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1098/2216] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:35 2022] POST /identity/v3/users => generated 301 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:36.112581 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0740243-15e5-46a9-93e3-a6e5d5be0dc5 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:36.126000 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0740243-15e5-46a9-93e3-a6e5d5be0dc5 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:36.132335 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561704f-eeec-4945-a2a8-dacfd4522c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:36.133091 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561704f-eeec-4945-a2a8-dacfd4522c3f admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:36.133314 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561704f-eeec-4945-a2a8-dacfd4522c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:36.133599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561704f-eeec-4945-a2a8-dacfd4522c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:36.134489 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561704f-eeec-4945-a2a8-dacfd4522c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:36.136176 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561704f-eeec-4945-a2a8-dacfd4522c3f admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:36.137836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1119/2217] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:35 2022] POST /identity/v3/users => generated 285 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:36.141892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09733062-a1d8-45af-8894-447282cc572a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:36.152829 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1099/2218] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:06:36.154299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09733062-a1d8-45af-8894-447282cc572a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:36.157062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561704f-eeec-4945-a2a8-dacfd4522c3f admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:36.160781 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c45b9-c9df-40c4-8d62-54fcba92b6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:36.161874 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c45b9-c9df-40c4-8d62-54fcba92b6d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:36.162385 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c45b9-c9df-40c4-8d62-54fcba92b6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:36.162945 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c45b9-c9df-40c4-8d62-54fcba92b6d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:36.163957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c45b9-c9df-40c4-8d62-54fcba92b6d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:36.166019 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c45b9-c9df-40c4-8d62-54fcba92b6d6 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:36.169962 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561704f-eeec-4945-a2a8-dacfd4522c3f admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:36.177522 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3eae67-8b67-42cc-a802-2974cac38c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:36.178257 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3eae67-8b67-42cc-a802-2974cac38c16 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:36.178618 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3eae67-8b67-42cc-a802-2974cac38c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:36.178798 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3eae67-8b67-42cc-a802-2974cac38c16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:36.179574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3eae67-8b67-42cc-a802-2974cac38c16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:36.181311 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3eae67-8b67-42cc-a802-2974cac38c16 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:36.183190 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1120/2219] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:06:36.184342 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c45b9-c9df-40c4-8d62-54fcba92b6d6 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:36.195559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1100/2220] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:06:36.199911 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3eae67-8b67-42cc-a802-2974cac38c16 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:36.200408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c45b9-c9df-40c4-8d62-54fcba92b6d6 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:36.206771 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774536a5-f316-4ebb-b0f1-1d7c973b4b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:36.207457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774536a5-f316-4ebb-b0f1-1d7c973b4b78 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:36.207754 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774536a5-f316-4ebb-b0f1-1d7c973b4b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:36.208085 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774536a5-f316-4ebb-b0f1-1d7c973b4b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:36.208828 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774536a5-f316-4ebb-b0f1-1d7c973b4b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:36.210681 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774536a5-f316-4ebb-b0f1-1d7c973b4b78 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:36.215592 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3eae67-8b67-42cc-a802-2974cac38c16 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:36.222842 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e3c90-8811-4690-aa28-25e646ae53f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:36.224041 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e3c90-8811-4690-aa28-25e646ae53f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:36.224880 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e3c90-8811-4690-aa28-25e646ae53f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:36.224880 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e3c90-8811-4690-aa28-25e646ae53f3 admin admin] PATH_INFO: `/v3/projects/62b66421d3e24a62a5aa122073377a82/users/677bcb1b41d7472899f44c9c8f141f72/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:36.225779 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1121/2221] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:36.226873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774536a5-f316-4ebb-b0f1-1d7c973b4b78 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:36.240199 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774536a5-f316-4ebb-b0f1-1d7c973b4b78 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:36.243358 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e3c90-8811-4690-aa28-25e646ae53f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62b66421d3e24a62a5aa122073377a82, user_id=677bcb1b41d7472899f44c9c8f141f72, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:36.245497 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cbe99b-34df-4888-a05d-300c65e004bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:36.246531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e3c90-8811-4690-aa28-25e646ae53f3 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:36.246613 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cbe99b-34df-4888-a05d-300c65e004bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:36.246820 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cbe99b-34df-4888-a05d-300c65e004bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:36.247131 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cbe99b-34df-4888-a05d-300c65e004bc admin admin] PATH_INFO: `/v3/projects/68590393cacc4bf1bfe65c4343d5a1da/users/eadf0a646bb048818edef5d3c51c7b7f/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:36.263813 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1101/2222] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:36 2022] PUT /identity/v3/projects/62b66421d3e24a62a5aa122073377a82/users/677bcb1b41d7472899f44c9c8f141f72/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:36.266873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cbe99b-34df-4888-a05d-300c65e004bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=68590393cacc4bf1bfe65c4343d5a1da, user_id=eadf0a646bb048818edef5d3c51c7b7f, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:36.269591 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cbe99b-34df-4888-a05d-300c65e004bc admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:36.275700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2065992-02f9-4585-a758-93686b5bdc58 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:36.276239 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2065992-02f9-4585-a758-93686b5bdc58 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:36.276625 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2065992-02f9-4585-a758-93686b5bdc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:36.302225 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1122/2223] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:36 2022] PUT /identity/v3/projects/68590393cacc4bf1bfe65c4343d5a1da/users/eadf0a646bb048818edef5d3c51c7b7f/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:36.308455 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-f2065992-02f9-4585-a758-93686b5bdc58 None None] MFA Rules not processed for user `677bcb1b41d7472899f44c9c8f141f72`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:36.310833 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f2065992-02f9-4585-a758-93686b5bdc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:36.313482 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d09768-731f-4fa0-95b6-7ee3a62fd2bb None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:36.313836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d09768-731f-4fa0-95b6-7ee3a62fd2bb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:36.314207 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d09768-731f-4fa0-95b6-7ee3a62fd2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:36.338119 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:36.344434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f2d09768-731f-4fa0-95b6-7ee3a62fd2bb None None] MFA Rules not processed for user `eadf0a646bb048818edef5d3c51c7b7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:36.345675 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f2d09768-731f-4fa0-95b6-7ee3a62fd2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:36.406745 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1102/2224] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:06:36.417710 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2065992-02f9-4585-a758-93686b5bdc58 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:36.436946 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1123/2225] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:36 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:06:36.454412 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d09768-731f-4fa0-95b6-7ee3a62fd2bb None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:36.456203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f2d09768-731f-4fa0-95b6-7ee3a62fd2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:36.488757 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2065992-02f9-4585-a758-93686b5bdc58 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:36.494652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f0c50-2ca0-46f2-b621-a6cd21dab129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:36.495365 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f0c50-2ca0-46f2-b621-a6cd21dab129 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:36.495653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f0c50-2ca0-46f2-b621-a6cd21dab129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:36.495936 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f0c50-2ca0-46f2-b621-a6cd21dab129 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:36.497100 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f0c50-2ca0-46f2-b621-a6cd21dab129 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:36.499216 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f0c50-2ca0-46f2-b621-a6cd21dab129 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:36.547285 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1103/2226] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:36 2022] POST /identity/v3/projects => generated 401 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:36.554107 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f0c50-2ca0-46f2-b621-a6cd21dab129 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:36.554358 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d09768-731f-4fa0-95b6-7ee3a62fd2bb None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:36.573866 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:36.641993 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb7292-6acf-417a-ad4f-65c4e8d60a3d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f4c9fc949c4a2c951b755234848cc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:36.643187 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb7292-6acf-417a-ad4f-65c4e8d60a3d service glance] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:36.643579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb7292-6acf-417a-ad4f-65c4e8d60a3d service glance] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:36.644009 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb7292-6acf-417a-ad4f-65c4e8d60a3d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:36.658692 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f0c50-2ca0-46f2-b621-a6cd21dab129 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:36.675433 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc745b-b538-4528-82b9-c9f11fefdd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:36.676835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc745b-b538-4528-82b9-c9f11fefdd97 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:36.678338 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc745b-b538-4528-82b9-c9f11fefdd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:36.678635 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc745b-b538-4528-82b9-c9f11fefdd97 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:36.679828 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc745b-b538-4528-82b9-c9f11fefdd97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:36.694770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc745b-b538-4528-82b9-c9f11fefdd97 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:36.729906 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1104/2227] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:36 2022] POST /identity/v3/users => generated 307 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:06:36.735891 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc745b-b538-4528-82b9-c9f11fefdd97 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:36.762921 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc745b-b538-4528-82b9-c9f11fefdd97 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:36.772889 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fb7292-6acf-417a-ad4f-65c4e8d60a3d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:36.774930 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fb7292-6acf-417a-ad4f-65c4e8d60a3d service glance] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:36.776700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376cbf55-6735-4b5a-8fe5-c139de9326b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:36.777837 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376cbf55-6735-4b5a-8fe5-c139de9326b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:36.778322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376cbf55-6735-4b5a-8fe5-c139de9326b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:36.778644 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376cbf55-6735-4b5a-8fe5-c139de9326b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:36.780194 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376cbf55-6735-4b5a-8fe5-c139de9326b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:36.786510 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376cbf55-6735-4b5a-8fe5-c139de9326b4 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:36.789537 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1124/2228] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:36 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:06:36.811620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1105/2229] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:36.819036 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb7292-6acf-417a-ad4f-65c4e8d60a3d service glance] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:36.838658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb7292-6acf-417a-ad4f-65c4e8d60a3d service glance] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:36.845752 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c548bf-26db-4ac7-8b6a-560584990b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:36.846780 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c548bf-26db-4ac7-8b6a-560584990b4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:36.847059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c548bf-26db-4ac7-8b6a-560584990b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:36.847332 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c548bf-26db-4ac7-8b6a-560584990b4f admin admin] PATH_INFO: `/v3/projects/9bf103b88a5b4a3babf775c337a3b679/users/0b6f698a41804a029fa403f5157096bf/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:36.867908 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c548bf-26db-4ac7-8b6a-560584990b4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf103b88a5b4a3babf775c337a3b679, user_id=0b6f698a41804a029fa403f5157096bf, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:36.872312 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c548bf-26db-4ac7-8b6a-560584990b4f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:36.914059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1125/2230] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:36 2022] PUT /identity/v3/projects/9bf103b88a5b4a3babf775c337a3b679/users/0b6f698a41804a029fa403f5157096bf/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:36.917785 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376cbf55-6735-4b5a-8fe5-c139de9326b4 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:36.931755 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376cbf55-6735-4b5a-8fe5-c139de9326b4 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:36.991370 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:37.056772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc2479-7d8e-4ceb-b23e-5286e9ba3b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:37.057566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc2479-7d8e-4ceb-b23e-5286e9ba3b46 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:37.057921 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc2479-7d8e-4ceb-b23e-5286e9ba3b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:37.058441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc2479-7d8e-4ceb-b23e-5286e9ba3b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:37.059252 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc2479-7d8e-4ceb-b23e-5286e9ba3b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:37.069403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc2479-7d8e-4ceb-b23e-5286e9ba3b46 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:37.083382 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1106/2231] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:37.087663 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c548bf-26db-4ac7-8b6a-560584990b4f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:37.102997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c548bf-26db-4ac7-8b6a-560584990b4f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:37.108998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd246e-7edc-420d-a7c4-63a3c2994fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:37.111027 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd246e-7edc-420d-a7c4-63a3c2994fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:37.111316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd246e-7edc-420d-a7c4-63a3c2994fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:37.111631 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd246e-7edc-420d-a7c4-63a3c2994fb1 admin admin] PATH_INFO: `/v3/projects/9bf103b88a5b4a3babf775c337a3b679/users/0b6f698a41804a029fa403f5157096bf/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:37.114575 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd246e-7edc-420d-a7c4-63a3c2994fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf103b88a5b4a3babf775c337a3b679, user_id=0b6f698a41804a029fa403f5157096bf, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:37.117322 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd246e-7edc-420d-a7c4-63a3c2994fb1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:37.130872 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1126/2232] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:37 2022] PUT /identity/v3/projects/9bf103b88a5b4a3babf775c337a3b679/users/0b6f698a41804a029fa403f5157096bf/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:37.143425 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9eceae-4a5c-4d4a-bb53-8c75b9cf4cf6 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:37.143717 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9eceae-4a5c-4d4a-bb53-8c75b9cf4cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:37.144021 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9eceae-4a5c-4d4a-bb53-8c75b9cf4cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:37.181186 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-da9eceae-4a5c-4d4a-bb53-8c75b9cf4cf6 None None] MFA Rules not processed for user `0b6f698a41804a029fa403f5157096bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:37.183009 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-da9eceae-4a5c-4d4a-bb53-8c75b9cf4cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:37.335791 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1107/2233] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:37 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:37.348050 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd246e-7edc-420d-a7c4-63a3c2994fb1 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:37.349740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-26fd246e-7edc-420d-a7c4-63a3c2994fb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:37.419837 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd246e-7edc-420d-a7c4-63a3c2994fb1 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:37.425238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bb59b-81c6-42fe-a2f7-4251e85d5532 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:37.425893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bb59b-81c6-42fe-a2f7-4251e85d5532 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:37.426166 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bb59b-81c6-42fe-a2f7-4251e85d5532 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:37.426476 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bb59b-81c6-42fe-a2f7-4251e85d5532 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:37.446847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18bb59b-81c6-42fe-a2f7-4251e85d5532 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:37.448483 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18bb59b-81c6-42fe-a2f7-4251e85d5532 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:37.461753 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1127/2234] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:37 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:37.572217 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9eceae-4a5c-4d4a-bb53-8c75b9cf4cf6 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:37.588794 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9eceae-4a5c-4d4a-bb53-8c75b9cf4cf6 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:37.601003 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47c854-e1ff-4242-bd0f-3cd3de316776 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:37.602028 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47c854-e1ff-4242-bd0f-3cd3de316776 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:37.602635 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47c854-e1ff-4242-bd0f-3cd3de316776 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:37.603044 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47c854-e1ff-4242-bd0f-3cd3de316776 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:37.672304 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47c854-e1ff-4242-bd0f-3cd3de316776 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:37.673683 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47c854-e1ff-4242-bd0f-3cd3de316776 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:37.684077 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1108/2235] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:42.136635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bb59b-81c6-42fe-a2f7-4251e85d5532 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:42.183757 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:42.213727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bb59b-81c6-42fe-a2f7-4251e85d5532 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:42.218695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101e6fb-8ed6-4853-b292-5ffbd9d2855c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:42.219612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101e6fb-8ed6-4853-b292-5ffbd9d2855c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:42.219895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101e6fb-8ed6-4853-b292-5ffbd9d2855c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:42.220169 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101e6fb-8ed6-4853-b292-5ffbd9d2855c admin admin] PATH_INFO: `/v3/users/677bcb1b41d7472899f44c9c8f141f72` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:42.221387 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101e6fb-8ed6-4853-b292-5ffbd9d2855c admin admin] RBAC: Authorizing `identity:delete_user(user_id=677bcb1b41d7472899f44c9c8f141f72)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:42.223445 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101e6fb-8ed6-4853-b292-5ffbd9d2855c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:42.274940 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2101e6fb-8ed6-4853-b292-5ffbd9d2855c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '677bcb1b41d7472899f44c9c8f141f72'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:42.283259 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2101e6fb-8ed6-4853-b292-5ffbd9d2855c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '677bcb1b41d7472899f44c9c8f141f72'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:42.300531 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2101e6fb-8ed6-4853-b292-5ffbd9d2855c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '677bcb1b41d7472899f44c9c8f141f72'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:42.301381 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2101e6fb-8ed6-4853-b292-5ffbd9d2855c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '677bcb1b41d7472899f44c9c8f141f72'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:42.302401 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2101e6fb-8ed6-4853-b292-5ffbd9d2855c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '677bcb1b41d7472899f44c9c8f141f72'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:42.318407 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1128/2236] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:42 2022] DELETE /identity/v3/users/677bcb1b41d7472899f44c9c8f141f72 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:42.323032 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5671d-a755-4c96-a668-db7a3429bad0 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:42.323580 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5671d-a755-4c96-a668-db7a3429bad0 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:42.323580 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5671d-a755-4c96-a668-db7a3429bad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:42.344094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-6de5671d-a755-4c96-a668-db7a3429bad0 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:42.345321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-6de5671d-a755-4c96-a668-db7a3429bad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:42.388455 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:42.409615 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1109/2237] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:06:42.425105 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101e6fb-8ed6-4853-b292-5ffbd9d2855c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:42.426655 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2101e6fb-8ed6-4853-b292-5ffbd9d2855c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:42.494516 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101e6fb-8ed6-4853-b292-5ffbd9d2855c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:42.505280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544e4de-baab-4e59-95cc-14f5a7f00f54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:42.506102 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544e4de-baab-4e59-95cc-14f5a7f00f54 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:42.506478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544e4de-baab-4e59-95cc-14f5a7f00f54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:42.506785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544e4de-baab-4e59-95cc-14f5a7f00f54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:42.526056 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7544e4de-baab-4e59-95cc-14f5a7f00f54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:42.527892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7544e4de-baab-4e59-95cc-14f5a7f00f54 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:42.539833 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1129/2238] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:06:42.921662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5671d-a755-4c96-a668-db7a3429bad0 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:42.923244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-6de5671d-a755-4c96-a668-db7a3429bad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:42.948347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5671d-a755-4c96-a668-db7a3429bad0 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:42.955228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03961596-945b-4de7-b577-28dc50bd0894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:42.956336 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03961596-945b-4de7-b577-28dc50bd0894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:42.956797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03961596-945b-4de7-b577-28dc50bd0894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:42.957238 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03961596-945b-4de7-b577-28dc50bd0894 admin admin] PATH_INFO: `/v3/projects/62b66421d3e24a62a5aa122073377a82` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:42.959305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03961596-945b-4de7-b577-28dc50bd0894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62b66421d3e24a62a5aa122073377a82)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:42.962259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03961596-945b-4de7-b577-28dc50bd0894 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:43.023754 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-03961596-945b-4de7-b577-28dc50bd0894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62b66421d3e24a62a5aa122073377a82', 'request_id': 'req-03961596-945b-4de7-b577-28dc50bd0894'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:43.031946 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-03961596-945b-4de7-b577-28dc50bd0894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62b66421d3e24a62a5aa122073377a82', 'request_id': 'req-03961596-945b-4de7-b577-28dc50bd0894'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:43.039128 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-03961596-945b-4de7-b577-28dc50bd0894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62b66421d3e24a62a5aa122073377a82', 'request_id': 'req-03961596-945b-4de7-b577-28dc50bd0894'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:43.054627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-03961596-945b-4de7-b577-28dc50bd0894 admin admin] The token cache is being invalidate because project 62b66421d3e24a62a5aa122073377a82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:43.054922 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-03961596-945b-4de7-b577-28dc50bd0894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:43.055581 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-03961596-945b-4de7-b577-28dc50bd0894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:43.057219 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1110/2239] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:42 2022] DELETE /identity/v3/projects/62b66421d3e24a62a5aa122073377a82 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:43.061404 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544e4de-baab-4e59-95cc-14f5a7f00f54 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:43.072353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544e4de-baab-4e59-95cc-14f5a7f00f54 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:43.152117 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:43.174515 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10556b2d-4c9e-4cab-b4ac-9f22e5a476ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:43.176985 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10556b2d-4c9e-4cab-b4ac-9f22e5a476ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:43.177287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10556b2d-4c9e-4cab-b4ac-9f22e5a476ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:43.177585 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10556b2d-4c9e-4cab-b4ac-9f22e5a476ca admin admin] PATH_INFO: `/v3/users/0b6f698a41804a029fa403f5157096bf` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:43.183459 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10556b2d-4c9e-4cab-b4ac-9f22e5a476ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b6f698a41804a029fa403f5157096bf)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:43.186989 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10556b2d-4c9e-4cab-b4ac-9f22e5a476ca admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:43.247349 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-10556b2d-4c9e-4cab-b4ac-9f22e5a476ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b6f698a41804a029fa403f5157096bf'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:43.258403 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-10556b2d-4c9e-4cab-b4ac-9f22e5a476ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b6f698a41804a029fa403f5157096bf'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:43.271725 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-10556b2d-4c9e-4cab-b4ac-9f22e5a476ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b6f698a41804a029fa403f5157096bf'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:43.272545 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-10556b2d-4c9e-4cab-b4ac-9f22e5a476ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b6f698a41804a029fa403f5157096bf'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:43.273638 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-10556b2d-4c9e-4cab-b4ac-9f22e5a476ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b6f698a41804a029fa403f5157096bf'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:43.288118 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1130/2240] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:43 2022] DELETE /identity/v3/users/0b6f698a41804a029fa403f5157096bf => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:43.724693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03961596-945b-4de7-b577-28dc50bd0894 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:43.728958 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-03961596-945b-4de7-b577-28dc50bd0894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:43.750615 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03961596-945b-4de7-b577-28dc50bd0894 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:43.854602 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a246d-2623-4e7d-ae08-5e7c726ca258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:43.855524 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a246d-2623-4e7d-ae08-5e7c726ca258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:43.855809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a246d-2623-4e7d-ae08-5e7c726ca258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:43.856085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a246d-2623-4e7d-ae08-5e7c726ca258 admin admin] PATH_INFO: `/v3/projects/9bf103b88a5b4a3babf775c337a3b679` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:43.857305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a246d-2623-4e7d-ae08-5e7c726ca258 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bf103b88a5b4a3babf775c337a3b679)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:43.859359 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a246d-2623-4e7d-ae08-5e7c726ca258 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:43.932500 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-435a246d-2623-4e7d-ae08-5e7c726ca258 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bf103b88a5b4a3babf775c337a3b679', 'request_id': 'req-435a246d-2623-4e7d-ae08-5e7c726ca258'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:43.940954 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-435a246d-2623-4e7d-ae08-5e7c726ca258 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bf103b88a5b4a3babf775c337a3b679', 'request_id': 'req-435a246d-2623-4e7d-ae08-5e7c726ca258'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:43.947968 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-435a246d-2623-4e7d-ae08-5e7c726ca258 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bf103b88a5b4a3babf775c337a3b679', 'request_id': 'req-435a246d-2623-4e7d-ae08-5e7c726ca258'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:43.957431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:43.963228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-435a246d-2623-4e7d-ae08-5e7c726ca258 admin admin] The token cache is being invalidate because project 9bf103b88a5b4a3babf775c337a3b679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:43.963613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-435a246d-2623-4e7d-ae08-5e7c726ca258 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:43.964560 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-435a246d-2623-4e7d-ae08-5e7c726ca258 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:43.967093 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1111/2241] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:43 2022] DELETE /identity/v3/projects/9bf103b88a5b4a3babf775c337a3b679 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:43.973063 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cc533-d531-4356-b8b2-073324ef75c3 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:43.973373 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cc533-d531-4356-b8b2-073324ef75c3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:43.973699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cc533-d531-4356-b8b2-073324ef75c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:44.008673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-768cc533-d531-4356-b8b2-073324ef75c3 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:44.010010 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-768cc533-d531-4356-b8b2-073324ef75c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:44.076131 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1131/2242] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:44.085727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c54f8-9392-482d-be40-efabc39de75a None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:44.086118 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c54f8-9392-482d-be40-efabc39de75a None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:44.086512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c54f8-9392-482d-be40-efabc39de75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:44.106528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-646c54f8-9392-482d-be40-efabc39de75a None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:44.107996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-646c54f8-9392-482d-be40-efabc39de75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:44.117253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1112/2243] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:44.121772 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768cc533-d531-4356-b8b2-073324ef75c3 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:44.133347 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768cc533-d531-4356-b8b2-073324ef75c3 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:44.143099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ffaed7-e919-4533-8c2d-606d9e80a4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:44.143099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffaed7-e919-4533-8c2d-606d9e80a4e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:44.143099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffaed7-e919-4533-8c2d-606d9e80a4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:44.143099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffaed7-e919-4533-8c2d-606d9e80a4e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:44.143536 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ffaed7-e919-4533-8c2d-606d9e80a4e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:06:44.143536 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ffaed7-e919-4533-8c2d-606d9e80a4e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:44.144231 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ffaed7-e919-4533-8c2d-606d9e80a4e3 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:44.161606 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1132/2244] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:06:44.166022 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c54f8-9392-482d-be40-efabc39de75a None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:44.179175 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c54f8-9392-482d-be40-efabc39de75a None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:44.187462 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c061b83-5a52-4930-9adc-0f560eb241b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:44.189580 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c061b83-5a52-4930-9adc-0f560eb241b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:44.189862 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c061b83-5a52-4930-9adc-0f560eb241b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:44.190244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c061b83-5a52-4930-9adc-0f560eb241b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:44.191086 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c061b83-5a52-4930-9adc-0f560eb241b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:44.193111 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c061b83-5a52-4930-9adc-0f560eb241b9 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:44.256920 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1113/2245] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:44 2022] POST /identity/v3/projects => generated 389 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:44.258455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ffaed7-e919-4533-8c2d-606d9e80a4e3 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:44.271320 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ffaed7-e919-4533-8c2d-606d9e80a4e3 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:44.368069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:44.378664 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b5c52-0a04-4768-b921-61c673254810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:44.386081 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b5c52-0a04-4768-b921-61c673254810 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:44.389547 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b5c52-0a04-4768-b921-61c673254810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:44.390079 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b5c52-0a04-4768-b921-61c673254810 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:44.391452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3b5c52-0a04-4768-b921-61c673254810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:44.396508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3b5c52-0a04-4768-b921-61c673254810 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:44.427094 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1133/2246] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:44 2022] POST /identity/v3/users => generated 295 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:44.430177 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c061b83-5a52-4930-9adc-0f560eb241b9 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:44.441657 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c061b83-5a52-4930-9adc-0f560eb241b9 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:44.448091 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bcc7fd-4274-4aba-aa3d-fff69e9dcef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:44.448794 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bcc7fd-4274-4aba-aa3d-fff69e9dcef9 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:44.449088 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bcc7fd-4274-4aba-aa3d-fff69e9dcef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:44.449368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bcc7fd-4274-4aba-aa3d-fff69e9dcef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:44.450118 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bcc7fd-4274-4aba-aa3d-fff69e9dcef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:44.452322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bcc7fd-4274-4aba-aa3d-fff69e9dcef9 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:44.475527 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1114/2247] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:44.479676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b5c52-0a04-4768-b921-61c673254810 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:44.491938 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b5c52-0a04-4768-b921-61c673254810 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:44.502876 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f0a1bc-d3a2-4d91-b221-bc8cacae72dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:44.503570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0a1bc-d3a2-4d91-b221-bc8cacae72dc admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:44.503855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0a1bc-d3a2-4d91-b221-bc8cacae72dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:44.504134 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0a1bc-d3a2-4d91-b221-bc8cacae72dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:44.505010 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f0a1bc-d3a2-4d91-b221-bc8cacae72dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:44.506852 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f0a1bc-d3a2-4d91-b221-bc8cacae72dc admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:44.521995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1134/2248] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:06:44.525693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bcc7fd-4274-4aba-aa3d-fff69e9dcef9 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:44.538954 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bcc7fd-4274-4aba-aa3d-fff69e9dcef9 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:44.544032 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c499d-9bdd-46f4-9001-341ae6e9cab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:44.544948 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c499d-9bdd-46f4-9001-341ae6e9cab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:44.545768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c499d-9bdd-46f4-9001-341ae6e9cab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:44.546042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c499d-9bdd-46f4-9001-341ae6e9cab1 admin admin] PATH_INFO: `/v3/projects/b2f684925a9c40c4bda4c474fee49c58/users/8d7c8d36812c49848baa1dccd38d6b68/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:44.571479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c499d-9bdd-46f4-9001-341ae6e9cab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2f684925a9c40c4bda4c474fee49c58, user_id=8d7c8d36812c49848baa1dccd38d6b68, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:44.574933 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c499d-9bdd-46f4-9001-341ae6e9cab1 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:44.603254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1115/2249] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:44 2022] PUT /identity/v3/projects/b2f684925a9c40c4bda4c474fee49c58/users/8d7c8d36812c49848baa1dccd38d6b68/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:44.617286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467312bf-d47b-430c-b148-ce59c4e2b342 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:44.617574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467312bf-d47b-430c-b148-ce59c4e2b342 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:44.617905 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467312bf-d47b-430c-b148-ce59c4e2b342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:44.650322 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-467312bf-d47b-430c-b148-ce59c4e2b342 None None] MFA Rules not processed for user `8d7c8d36812c49848baa1dccd38d6b68`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:44.651574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-467312bf-d47b-430c-b148-ce59c4e2b342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:44.727736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1135/2250] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:44.733732 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c499d-9bdd-46f4-9001-341ae6e9cab1 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:44.804058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c499d-9bdd-46f4-9001-341ae6e9cab1 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:44.809675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b0127-d488-4951-9b4b-f4fb6d2635f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:44.810483 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b0127-d488-4951-9b4b-f4fb6d2635f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:44.810796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b0127-d488-4951-9b4b-f4fb6d2635f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:44.811196 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b0127-d488-4951-9b4b-f4fb6d2635f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:44.811914 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b0127-d488-4951-9b4b-f4fb6d2635f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:44.813899 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b0127-d488-4951-9b4b-f4fb6d2635f2 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:44.847501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1116/2251] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:44 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:44.851334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467312bf-d47b-430c-b148-ce59c4e2b342 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:44.879867 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467312bf-d47b-430c-b148-ce59c4e2b342 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:44.881365 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b0127-d488-4951-9b4b-f4fb6d2635f2 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:44.882778 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f79b0127-d488-4951-9b4b-f4fb6d2635f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:44.949676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33140780-76b0-44a8-971f-d23f4ed85b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:44.950423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33140780-76b0-44a8-971f-d23f4ed85b6d admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:44.950714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33140780-76b0-44a8-971f-d23f4ed85b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:44.950996 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33140780-76b0-44a8-971f-d23f4ed85b6d admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:44.952465 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33140780-76b0-44a8-971f-d23f4ed85b6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:44.954485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33140780-76b0-44a8-971f-d23f4ed85b6d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:44.961244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b0127-d488-4951-9b4b-f4fb6d2635f2 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:44.967118 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bcdad4-42ca-4dd1-86d9-ddeec47ecd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:44.968855 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcdad4-42ca-4dd1-86d9-ddeec47ecd34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:44.969135 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcdad4-42ca-4dd1-86d9-ddeec47ecd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:44.969449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcdad4-42ca-4dd1-86d9-ddeec47ecd34 admin admin] PATH_INFO: `/v3/users/c3e1ac754d9843a881835442cade5d1e` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:44.970777 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bcdad4-42ca-4dd1-86d9-ddeec47ecd34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3e1ac754d9843a881835442cade5d1e)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:44.972725 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bcdad4-42ca-4dd1-86d9-ddeec47ecd34 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:44.976211 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1136/2252] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:44 2022] POST /identity/v3/users => generated 300 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:44.980021 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33140780-76b0-44a8-971f-d23f4ed85b6d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:44.990248 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33140780-76b0-44a8-971f-d23f4ed85b6d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:44.995137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0e6c6-bb3c-4c90-a82c-438598144ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:44.995804 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0e6c6-bb3c-4c90-a82c-438598144ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:44.996092 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0e6c6-bb3c-4c90-a82c-438598144ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:44.996613 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0e6c6-bb3c-4c90-a82c-438598144ef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:44.996889 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:44.997430 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f0e6c6-bb3c-4c90-a82c-438598144ef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:45.000002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f0e6c6-bb3c-4c90-a82c-438598144ef9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:45.016352 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1137/2253] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:45.020899 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0e6c6-bb3c-4c90-a82c-438598144ef9 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:45.035236 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0e6c6-bb3c-4c90-a82c-438598144ef9 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:45.039973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42169218-d0b5-40d8-9f3b-50f56acd42c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:45.040868 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42169218-d0b5-40d8-9f3b-50f56acd42c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:45.041155 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42169218-d0b5-40d8-9f3b-50f56acd42c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:45.041431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42169218-d0b5-40d8-9f3b-50f56acd42c0 admin admin] PATH_INFO: `/v3/projects/42bf4a4d85a04bd7b2c6fc0166fa4c44/users/de9d309b438745e582965841df350bf2/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:45.042657 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-a9bcdad4-42ca-4dd1-86d9-ddeec47ecd34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3e1ac754d9843a881835442cade5d1e'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:45.050667 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-a9bcdad4-42ca-4dd1-86d9-ddeec47ecd34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3e1ac754d9843a881835442cade5d1e'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:45.051779 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-a9bcdad4-42ca-4dd1-86d9-ddeec47ecd34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3e1ac754d9843a881835442cade5d1e'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:45.052655 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-a9bcdad4-42ca-4dd1-86d9-ddeec47ecd34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3e1ac754d9843a881835442cade5d1e'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:45.061363 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42169218-d0b5-40d8-9f3b-50f56acd42c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42bf4a4d85a04bd7b2c6fc0166fa4c44, user_id=de9d309b438745e582965841df350bf2, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:45.064029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42169218-d0b5-40d8-9f3b-50f56acd42c0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:45.069008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-a9bcdad4-42ca-4dd1-86d9-ddeec47ecd34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3e1ac754d9843a881835442cade5d1e'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:45.079801 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1138/2254] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:45 2022] PUT /identity/v3/projects/42bf4a4d85a04bd7b2c6fc0166fa4c44/users/de9d309b438745e582965841df350bf2/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:45.083561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42169218-d0b5-40d8-9f3b-50f56acd42c0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:45.085193 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-42169218-d0b5-40d8-9f3b-50f56acd42c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:45.088998 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1117/2255] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:44 2022] DELETE /identity/v3/users/c3e1ac754d9843a881835442cade5d1e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:06:45.094220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b38ca7-7edb-4032-9bef-45f73907348e None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:45.094523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b38ca7-7edb-4032-9bef-45f73907348e None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:45.094825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b38ca7-7edb-4032-9bef-45f73907348e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:45.109516 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:45.114669 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-83b38ca7-7edb-4032-9bef-45f73907348e None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:45.115953 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-83b38ca7-7edb-4032-9bef-45f73907348e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:45.167457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42169218-d0b5-40d8-9f3b-50f56acd42c0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:45.181277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1118/2256] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:06:45.191097 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b38ca7-7edb-4032-9bef-45f73907348e None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:45.192383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-83b38ca7-7edb-4032-9bef-45f73907348e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:45.227556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960aaa88-cc24-4eaa-9598-fcba3f3f80ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:45.228265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960aaa88-cc24-4eaa-9598-fcba3f3f80ad admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:45.228555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960aaa88-cc24-4eaa-9598-fcba3f3f80ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:45.228838 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960aaa88-cc24-4eaa-9598-fcba3f3f80ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:45.229953 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960aaa88-cc24-4eaa-9598-fcba3f3f80ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:45.231210 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960aaa88-cc24-4eaa-9598-fcba3f3f80ad admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:45.243776 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1139/2257] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:06:45.247849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960aaa88-cc24-4eaa-9598-fcba3f3f80ad admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:45.259816 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960aaa88-cc24-4eaa-9598-fcba3f3f80ad admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:45.265851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a8602-867a-4973-b0e6-3e4668c23b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:45.266944 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a8602-867a-4973-b0e6-3e4668c23b04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:45.267385 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a8602-867a-4973-b0e6-3e4668c23b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:45.267686 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a8602-867a-4973-b0e6-3e4668c23b04 admin admin] PATH_INFO: `/v3/projects/42bf4a4d85a04bd7b2c6fc0166fa4c44/users/de9d309b438745e582965841df350bf2/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:45.271107 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a8602-867a-4973-b0e6-3e4668c23b04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42bf4a4d85a04bd7b2c6fc0166fa4c44, user_id=de9d309b438745e582965841df350bf2, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:45.273786 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a8602-867a-4973-b0e6-3e4668c23b04 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:45.286327 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b38ca7-7edb-4032-9bef-45f73907348e None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:45.288240 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1140/2258] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:45 2022] PUT /identity/v3/projects/42bf4a4d85a04bd7b2c6fc0166fa4c44/users/de9d309b438745e582965841df350bf2/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:45.300535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d9072-4d36-459e-872d-5b6a1797e5de None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:45.300834 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d9072-4d36-459e-872d-5b6a1797e5de None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:45.301117 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d9072-4d36-459e-872d-5b6a1797e5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:45.351734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d97d9072-4d36-459e-872d-5b6a1797e5de None None] MFA Rules not processed for user `de9d309b438745e582965841df350bf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:45.358591 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d97d9072-4d36-459e-872d-5b6a1797e5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:45.385327 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a05d2f-58b6-4f0b-8457-f7ea13ae7d88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:45.385984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a05d2f-58b6-4f0b-8457-f7ea13ae7d88 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:45.386305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a05d2f-58b6-4f0b-8457-f7ea13ae7d88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:45.386585 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a05d2f-58b6-4f0b-8457-f7ea13ae7d88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:45.400850 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:45.449531 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1141/2259] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:06:45.467652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a05d2f-58b6-4f0b-8457-f7ea13ae7d88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:45.467652 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a05d2f-58b6-4f0b-8457-f7ea13ae7d88 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:45.468707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d9072-4d36-459e-872d-5b6a1797e5de None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:45.470073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d97d9072-4d36-459e-872d-5b6a1797e5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:45.477243 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1119/2260] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:06:45.506221 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:45.541325 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d9072-4d36-459e-872d-5b6a1797e5de None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:45.543835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87114f40-cd0a-4394-8289-3e0fa9329681 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:45.544543 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87114f40-cd0a-4394-8289-3e0fa9329681 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:45.544830 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87114f40-cd0a-4394-8289-3e0fa9329681 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:45.545111 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87114f40-cd0a-4394-8289-3e0fa9329681 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:45.556558 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87114f40-cd0a-4394-8289-3e0fa9329681 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:45.558118 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87114f40-cd0a-4394-8289-3e0fa9329681 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:45.569733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1142/2261] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:06:45.761292 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a05d2f-58b6-4f0b-8457-f7ea13ae7d88 service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:45.781103 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a05d2f-58b6-4f0b-8457-f7ea13ae7d88 service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:45.786710 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25238e8c-62cf-4d2a-99f1-2c22c4236ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:45.787648 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25238e8c-62cf-4d2a-99f1-2c22c4236ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:45.787933 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25238e8c-62cf-4d2a-99f1-2c22c4236ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:45.788206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25238e8c-62cf-4d2a-99f1-2c22c4236ed5 admin admin] PATH_INFO: `/v3/users/8d7c8d36812c49848baa1dccd38d6b68` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:45.789559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25238e8c-62cf-4d2a-99f1-2c22c4236ed5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d7c8d36812c49848baa1dccd38d6b68)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:45.792543 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25238e8c-62cf-4d2a-99f1-2c22c4236ed5 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:45.893470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87114f40-cd0a-4394-8289-3e0fa9329681 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:45.895003 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-87114f40-cd0a-4394-8289-3e0fa9329681 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:45.910883 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-25238e8c-62cf-4d2a-99f1-2c22c4236ed5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d7c8d36812c49848baa1dccd38d6b68'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:45.930318 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87114f40-cd0a-4394-8289-3e0fa9329681 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:45.933576 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-25238e8c-62cf-4d2a-99f1-2c22c4236ed5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d7c8d36812c49848baa1dccd38d6b68'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:45.935532 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687eb6be-4f2a-421e-a225-63c8ce98fca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:45.935898 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-25238e8c-62cf-4d2a-99f1-2c22c4236ed5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d7c8d36812c49848baa1dccd38d6b68'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:45.937853 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687eb6be-4f2a-421e-a225-63c8ce98fca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:45.937853 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687eb6be-4f2a-421e-a225-63c8ce98fca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:45.938061 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687eb6be-4f2a-421e-a225-63c8ce98fca9 admin admin] PATH_INFO: `/v3/projects/71fe7e98fefc4471be3cb40564a67411` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:45.938275 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-25238e8c-62cf-4d2a-99f1-2c22c4236ed5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d7c8d36812c49848baa1dccd38d6b68'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:45.939338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687eb6be-4f2a-421e-a225-63c8ce98fca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71fe7e98fefc4471be3cb40564a67411)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:45.941974 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687eb6be-4f2a-421e-a225-63c8ce98fca9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:45.956917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-25238e8c-62cf-4d2a-99f1-2c22c4236ed5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d7c8d36812c49848baa1dccd38d6b68'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:45.986239 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1120/2262] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:45 2022] DELETE /identity/v3/users/8d7c8d36812c49848baa1dccd38d6b68 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:45.990293 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cd684-a8e6-491e-b22d-9e4740eba1e0 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:45.990871 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cd684-a8e6-491e-b22d-9e4740eba1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:45.991410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cd684-a8e6-491e-b22d-9e4740eba1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:46.002495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-687eb6be-4f2a-421e-a225-63c8ce98fca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71fe7e98fefc4471be3cb40564a67411', 'request_id': 'req-687eb6be-4f2a-421e-a225-63c8ce98fca9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:46.011066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-687eb6be-4f2a-421e-a225-63c8ce98fca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71fe7e98fefc4471be3cb40564a67411', 'request_id': 'req-687eb6be-4f2a-421e-a225-63c8ce98fca9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:46.013496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-f88cd684-a8e6-491e-b22d-9e4740eba1e0 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:46.014835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f88cd684-a8e6-491e-b22d-9e4740eba1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:46.020671 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-687eb6be-4f2a-421e-a225-63c8ce98fca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71fe7e98fefc4471be3cb40564a67411', 'request_id': 'req-687eb6be-4f2a-421e-a225-63c8ce98fca9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:46.037866 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-687eb6be-4f2a-421e-a225-63c8ce98fca9 admin admin] The token cache is being invalidate because project 71fe7e98fefc4471be3cb40564a67411 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:46.038285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-687eb6be-4f2a-421e-a225-63c8ce98fca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:46.038883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-687eb6be-4f2a-421e-a225-63c8ce98fca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:46.040824 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1143/2263] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:45 2022] DELETE /identity/v3/projects/71fe7e98fefc4471be3cb40564a67411 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:46.046595 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07063d-d5d2-49ea-8b08-0eea0e25f10b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:46.046833 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07063d-d5d2-49ea-8b08-0eea0e25f10b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:46.047137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07063d-d5d2-49ea-8b08-0eea0e25f10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:46.088909 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6e07063d-d5d2-49ea-8b08-0eea0e25f10b None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:46.090497 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6e07063d-d5d2-49ea-8b08-0eea0e25f10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:46.132207 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1121/2264] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:46.144476 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88cd684-a8e6-491e-b22d-9e4740eba1e0 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:46.145885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f88cd684-a8e6-491e-b22d-9e4740eba1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:46.169709 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:46.176926 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1144/2265] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:06:46.185650 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ac223-db11-4411-8013-f3716e26b0bd None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:46.186001 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ac223-db11-4411-8013-f3716e26b0bd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:46.186338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ac223-db11-4411-8013-f3716e26b0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:46.204596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6e5ac223-db11-4411-8013-f3716e26b0bd None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:46.205807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6e5ac223-db11-4411-8013-f3716e26b0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:46.214609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88cd684-a8e6-491e-b22d-9e4740eba1e0 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:46.217810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1145/2266] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:06:46.218956 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ac223-db11-4411-8013-f3716e26b0bd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:46.219226 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfd995-a938-4b7d-b967-d9a6bfb5372b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:46.219881 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfd995-a938-4b7d-b967-d9a6bfb5372b service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:46.220170 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfd995-a938-4b7d-b967-d9a6bfb5372b service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:46.220456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfd995-a938-4b7d-b967-d9a6bfb5372b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:46.228798 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ac223-db11-4411-8013-f3716e26b0bd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:46.231197 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dfd995-a938-4b7d-b967-d9a6bfb5372b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:46.233504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dfd995-a938-4b7d-b967-d9a6bfb5372b service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:46.234579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c1a9b-27b1-4f3f-a80f-8707edcf3e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:46.235247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c1a9b-27b1-4f3f-a80f-8707edcf3e88 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:46.235533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c1a9b-27b1-4f3f-a80f-8707edcf3e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:46.235811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c1a9b-27b1-4f3f-a80f-8707edcf3e88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:46.236544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c1a9b-27b1-4f3f-a80f-8707edcf3e88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:06:46.236848 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c1a9b-27b1-4f3f-a80f-8707edcf3e88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:46.238539 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c1a9b-27b1-4f3f-a80f-8707edcf3e88 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:46.244274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1122/2267] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:06:46.255036 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1146/2268] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:06:46.256165 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfd995-a938-4b7d-b967-d9a6bfb5372b service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:46.266015 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfd995-a938-4b7d-b967-d9a6bfb5372b service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:46.271083 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54aa94b-825a-40d3-9e01-262ce22385cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:46.271736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54aa94b-825a-40d3-9e01-262ce22385cf admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:46.272026 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54aa94b-825a-40d3-9e01-262ce22385cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:46.272307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54aa94b-825a-40d3-9e01-262ce22385cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:46.273021 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54aa94b-825a-40d3-9e01-262ce22385cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:46.274969 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54aa94b-825a-40d3-9e01-262ce22385cf admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:46.306729 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1123/2269] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:46 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:06:46.311184 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c1a9b-27b1-4f3f-a80f-8707edcf3e88 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:46.324049 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c1a9b-27b1-4f3f-a80f-8707edcf3e88 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:46.368660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:46.396788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53bedd-4bbb-46a9-9c2c-e69971258fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:46.397484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53bedd-4bbb-46a9-9c2c-e69971258fbb admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:46.397770 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53bedd-4bbb-46a9-9c2c-e69971258fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:46.398044 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53bedd-4bbb-46a9-9c2c-e69971258fbb admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:46.398796 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe53bedd-4bbb-46a9-9c2c-e69971258fbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:46.400743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe53bedd-4bbb-46a9-9c2c-e69971258fbb admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:46.426260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1147/2270] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:46 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:46.428653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54aa94b-825a-40d3-9e01-262ce22385cf admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:46.439919 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54aa94b-825a-40d3-9e01-262ce22385cf admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:46.444606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80f284-d584-4842-ae9a-4f24e42fad55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:46.445279 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80f284-d584-4842-ae9a-4f24e42fad55 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:46.445568 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80f284-d584-4842-ae9a-4f24e42fad55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:46.445849 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80f284-d584-4842-ae9a-4f24e42fad55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:46.446624 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80f284-d584-4842-ae9a-4f24e42fad55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:46.448226 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80f284-d584-4842-ae9a-4f24e42fad55 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:46.462165 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1124/2271] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:46.466234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53bedd-4bbb-46a9-9c2c-e69971258fbb admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:46.477420 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53bedd-4bbb-46a9-9c2c-e69971258fbb admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:46.482692 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877459d-0c3a-49a7-92da-37d03cfc2d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:46.483500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877459d-0c3a-49a7-92da-37d03cfc2d85 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:46.483677 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877459d-0c3a-49a7-92da-37d03cfc2d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:46.484051 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877459d-0c3a-49a7-92da-37d03cfc2d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:46.484927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877459d-0c3a-49a7-92da-37d03cfc2d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:46.486561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877459d-0c3a-49a7-92da-37d03cfc2d85 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:46.497384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1148/2272] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:46.502991 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80f284-d584-4842-ae9a-4f24e42fad55 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:46.518260 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80f284-d584-4842-ae9a-4f24e42fad55 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:46.523486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708c2ad-31d6-4a51-8540-a3bb38c14fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:46.524532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708c2ad-31d6-4a51-8540-a3bb38c14fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:46.524778 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708c2ad-31d6-4a51-8540-a3bb38c14fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:46.525136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708c2ad-31d6-4a51-8540-a3bb38c14fc2 admin admin] PATH_INFO: `/v3/projects/45d94db7075c40cbae3f106a9661d969/users/42c96a85400941818b658cca8cb56fec/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:46.544363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708c2ad-31d6-4a51-8540-a3bb38c14fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d94db7075c40cbae3f106a9661d969, user_id=42c96a85400941818b658cca8cb56fec, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:46.549345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708c2ad-31d6-4a51-8540-a3bb38c14fc2 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:46.551141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877459d-0c3a-49a7-92da-37d03cfc2d85 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:46.553654 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c877459d-0c3a-49a7-92da-37d03cfc2d85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:46.571682 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877459d-0c3a-49a7-92da-37d03cfc2d85 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:46.580779 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1125/2273] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:46 2022] PUT /identity/v3/projects/45d94db7075c40cbae3f106a9661d969/users/42c96a85400941818b658cca8cb56fec/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:46.588344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0033b-77ab-4a41-82d3-8012e1cca919 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:46.588630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0033b-77ab-4a41-82d3-8012e1cca919 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:46.589235 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0033b-77ab-4a41-82d3-8012e1cca919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:46.608461 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-1bb0033b-77ab-4a41-82d3-8012e1cca919 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:46.609745 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-1bb0033b-77ab-4a41-82d3-8012e1cca919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:46.645121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7791be-abb5-42f9-a08b-b67a452150ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:46.646774 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7791be-abb5-42f9-a08b-b67a452150ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:46.647301 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7791be-abb5-42f9-a08b-b67a452150ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:46.647812 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7791be-abb5-42f9-a08b-b67a452150ec admin admin] PATH_INFO: `/v3/projects/b2f684925a9c40c4bda4c474fee49c58` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:46.650838 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7791be-abb5-42f9-a08b-b67a452150ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2f684925a9c40c4bda4c474fee49c58)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:46.652937 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7791be-abb5-42f9-a08b-b67a452150ec admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:46.686627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1126/2274] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:46.696165 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0033b-77ab-4a41-82d3-8012e1cca919 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:46.697551 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-1bb0033b-77ab-4a41-82d3-8012e1cca919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:46.705183 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8c7791be-abb5-42f9-a08b-b67a452150ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2f684925a9c40c4bda4c474fee49c58', 'request_id': 'req-8c7791be-abb5-42f9-a08b-b67a452150ec'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:46.710947 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8c7791be-abb5-42f9-a08b-b67a452150ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2f684925a9c40c4bda4c474fee49c58', 'request_id': 'req-8c7791be-abb5-42f9-a08b-b67a452150ec'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:46.718501 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8c7791be-abb5-42f9-a08b-b67a452150ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2f684925a9c40c4bda4c474fee49c58', 'request_id': 'req-8c7791be-abb5-42f9-a08b-b67a452150ec'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:46.732211 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8c7791be-abb5-42f9-a08b-b67a452150ec admin admin] The token cache is being invalidate because project b2f684925a9c40c4bda4c474fee49c58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:46.732497 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8c7791be-abb5-42f9-a08b-b67a452150ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:46.733344 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8c7791be-abb5-42f9-a08b-b67a452150ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:46.735801 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1149/2275] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:46 2022] DELETE /identity/v3/projects/b2f684925a9c40c4bda4c474fee49c58 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:06:46.739039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7791be-abb5-42f9-a08b-b67a452150ec admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:46.740513 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8c7791be-abb5-42f9-a08b-b67a452150ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:46.758050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0033b-77ab-4a41-82d3-8012e1cca919 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:46.775980 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:46.794155 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:46.814296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7791be-abb5-42f9-a08b-b67a452150ec admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:46.815328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c0ecd4-e11d-44ff-bfcb-d7fe1ad3ce9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:46.816467 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0ecd4-e11d-44ff-bfcb-d7fe1ad3ce9f service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:46.816740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0ecd4-e11d-44ff-bfcb-d7fe1ad3ce9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:46.816975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0ecd4-e11d-44ff-bfcb-d7fe1ad3ce9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:46.819017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-45c0ecd4-e11d-44ff-bfcb-d7fe1ad3ce9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:46.831791 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27508a79-0aa6-4369-9af2-8fffc02bb9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:46.833494 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27508a79-0aa6-4369-9af2-8fffc02bb9ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:46.834053 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27508a79-0aa6-4369-9af2-8fffc02bb9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:46.834812 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27508a79-0aa6-4369-9af2-8fffc02bb9ae admin admin] PATH_INFO: `/v3/users/de9d309b438745e582965841df350bf2` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:46.835174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0ecd4-e11d-44ff-bfcb-d7fe1ad3ce9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:46.836820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0ecd4-e11d-44ff-bfcb-d7fe1ad3ce9f service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:46.837378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27508a79-0aa6-4369-9af2-8fffc02bb9ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=de9d309b438745e582965841df350bf2)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:46.840330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27508a79-0aa6-4369-9af2-8fffc02bb9ae admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:46.848956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1127/2276] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:06:46.915832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-27508a79-0aa6-4369-9af2-8fffc02bb9ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de9d309b438745e582965841df350bf2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:46.928034 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-27508a79-0aa6-4369-9af2-8fffc02bb9ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de9d309b438745e582965841df350bf2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:46.942665 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-27508a79-0aa6-4369-9af2-8fffc02bb9ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de9d309b438745e582965841df350bf2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:46.943389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-27508a79-0aa6-4369-9af2-8fffc02bb9ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de9d309b438745e582965841df350bf2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:46.944295 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-27508a79-0aa6-4369-9af2-8fffc02bb9ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de9d309b438745e582965841df350bf2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:46.959002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1150/2277] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:46 2022] DELETE /identity/v3/users/de9d309b438745e582965841df350bf2 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:06:47.452097 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c0ecd4-e11d-44ff-bfcb-d7fe1ad3ce9f service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:47.477999 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c0ecd4-e11d-44ff-bfcb-d7fe1ad3ce9f service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:47.606542 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0a6f8-2cd7-4aee-bdfd-335458ad9ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:47.608480 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0a6f8-2cd7-4aee-bdfd-335458ad9ff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:47.609196 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0a6f8-2cd7-4aee-bdfd-335458ad9ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:47.609375 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0a6f8-2cd7-4aee-bdfd-335458ad9ff1 admin admin] PATH_INFO: `/v3/projects/42bf4a4d85a04bd7b2c6fc0166fa4c44` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:47.610851 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa0a6f8-2cd7-4aee-bdfd-335458ad9ff1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42bf4a4d85a04bd7b2c6fc0166fa4c44)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:47.613040 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa0a6f8-2cd7-4aee-bdfd-335458ad9ff1 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:47.696885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-daa0a6f8-2cd7-4aee-bdfd-335458ad9ff1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42bf4a4d85a04bd7b2c6fc0166fa4c44', 'request_id': 'req-daa0a6f8-2cd7-4aee-bdfd-335458ad9ff1'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:47.709683 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-daa0a6f8-2cd7-4aee-bdfd-335458ad9ff1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42bf4a4d85a04bd7b2c6fc0166fa4c44', 'request_id': 'req-daa0a6f8-2cd7-4aee-bdfd-335458ad9ff1'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:47.721409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-daa0a6f8-2cd7-4aee-bdfd-335458ad9ff1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42bf4a4d85a04bd7b2c6fc0166fa4c44', 'request_id': 'req-daa0a6f8-2cd7-4aee-bdfd-335458ad9ff1'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:47.747986 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-daa0a6f8-2cd7-4aee-bdfd-335458ad9ff1 admin admin] The token cache is being invalidate because project 42bf4a4d85a04bd7b2c6fc0166fa4c44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:06:47.748521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-daa0a6f8-2cd7-4aee-bdfd-335458ad9ff1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:47.749613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-daa0a6f8-2cd7-4aee-bdfd-335458ad9ff1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:47.757943 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63af01-c26b-4989-948a-214cc8480d38 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:47.758577 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63af01-c26b-4989-948a-214cc8480d38 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:47.758671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1128/2278] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:06:47 2022] DELETE /identity/v3/projects/42bf4a4d85a04bd7b2c6fc0166fa4c44 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:47.758763 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63af01-c26b-4989-948a-214cc8480d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:47.778604 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8a63af01-c26b-4989-948a-214cc8480d38 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:47.780553 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8a63af01-c26b-4989-948a-214cc8480d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:47.861760 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1151/2279] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:47.871272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b9ad7-1eaf-4945-b386-b1670cfbf772 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:47.871914 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b9ad7-1eaf-4945-b386-b1670cfbf772 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:47.872492 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b9ad7-1eaf-4945-b386-b1670cfbf772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:47.899195 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-4c0b9ad7-1eaf-4945-b386-b1670cfbf772 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:47.900459 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-4c0b9ad7-1eaf-4945-b386-b1670cfbf772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:47.914705 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1129/2280] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:47.920060 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63af01-c26b-4989-948a-214cc8480d38 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:47.947828 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63af01-c26b-4989-948a-214cc8480d38 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:47.956810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc380117-fcb6-40dc-b80b-8b6ec65038dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:47.958231 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc380117-fcb6-40dc-b80b-8b6ec65038dc admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:47.958867 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc380117-fcb6-40dc-b80b-8b6ec65038dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:47.959566 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc380117-fcb6-40dc-b80b-8b6ec65038dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:47.961018 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc380117-fcb6-40dc-b80b-8b6ec65038dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:06:47.961461 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc380117-fcb6-40dc-b80b-8b6ec65038dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:47.963730 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc380117-fcb6-40dc-b80b-8b6ec65038dc admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:47.999842 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1152/2281] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:06:48.003941 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b9ad7-1eaf-4945-b386-b1670cfbf772 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:48.021490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b9ad7-1eaf-4945-b386-b1670cfbf772 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:48.025964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0ecdd-d4ec-4fd4-aaaf-da4a1cf3a440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:48.026712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0ecdd-d4ec-4fd4-aaaf-da4a1cf3a440 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:48.026995 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0ecdd-d4ec-4fd4-aaaf-da4a1cf3a440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:48.027272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0ecdd-d4ec-4fd4-aaaf-da4a1cf3a440 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:48.029330 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0ecdd-d4ec-4fd4-aaaf-da4a1cf3a440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:48.031504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0ecdd-d4ec-4fd4-aaaf-da4a1cf3a440 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:48.097205 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1130/2282] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:48 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:48.097879 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc380117-fcb6-40dc-b80b-8b6ec65038dc admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:48.108790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc380117-fcb6-40dc-b80b-8b6ec65038dc admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:48.199666 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:48.213061 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df1017-e8fa-485a-8168-4b7953c6f395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:48.214065 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df1017-e8fa-485a-8168-4b7953c6f395 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:48.214549 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df1017-e8fa-485a-8168-4b7953c6f395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:48.214978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df1017-e8fa-485a-8168-4b7953c6f395 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:48.215930 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df1017-e8fa-485a-8168-4b7953c6f395 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:48.218914 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df1017-e8fa-485a-8168-4b7953c6f395 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:48.263384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1153/2283] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:48 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:06:48.268347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0ecdd-d4ec-4fd4-aaaf-da4a1cf3a440 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:48.282665 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0ecdd-d4ec-4fd4-aaaf-da4a1cf3a440 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:48.294143 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24137a-72ae-4e55-8a41-5bf81e72d9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:48.295343 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24137a-72ae-4e55-8a41-5bf81e72d9d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:48.296063 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24137a-72ae-4e55-8a41-5bf81e72d9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:48.296508 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24137a-72ae-4e55-8a41-5bf81e72d9d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:48.298912 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24137a-72ae-4e55-8a41-5bf81e72d9d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:48.306445 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24137a-72ae-4e55-8a41-5bf81e72d9d1 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:48.328491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1131/2284] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:48.332444 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df1017-e8fa-485a-8168-4b7953c6f395 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:48.343403 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df1017-e8fa-485a-8168-4b7953c6f395 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:48.359778 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5d887-a6f1-4b67-80de-16452f57b59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:48.360791 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5d887-a6f1-4b67-80de-16452f57b59b admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:48.361293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5d887-a6f1-4b67-80de-16452f57b59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:48.361871 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5d887-a6f1-4b67-80de-16452f57b59b admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:48.363504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5d887-a6f1-4b67-80de-16452f57b59b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:48.367016 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5d887-a6f1-4b67-80de-16452f57b59b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:48.385108 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1154/2285] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:48.387007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24137a-72ae-4e55-8a41-5bf81e72d9d1 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:48.401021 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24137a-72ae-4e55-8a41-5bf81e72d9d1 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:48.409658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52664e5-b069-4b4f-9165-806e6f3be472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:48.410665 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52664e5-b069-4b4f-9165-806e6f3be472 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:48.411676 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52664e5-b069-4b4f-9165-806e6f3be472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:48.411966 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52664e5-b069-4b4f-9165-806e6f3be472 admin admin] PATH_INFO: `/v3/projects/cf1c9dce6a9c48aeaff0d0e503f72078/users/e20c26710df6481288e8746a74410708/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:48.426357 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:48.435108 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52664e5-b069-4b4f-9165-806e6f3be472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf1c9dce6a9c48aeaff0d0e503f72078, user_id=e20c26710df6481288e8746a74410708, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:48.439332 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52664e5-b069-4b4f-9165-806e6f3be472 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:48.465724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1132/2286] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:48 2022] PUT /identity/v3/projects/cf1c9dce6a9c48aeaff0d0e503f72078/users/e20c26710df6481288e8746a74410708/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:48.476958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3373ad-4fef-4d51-b135-da77275d4641 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:48.477301 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3373ad-4fef-4d51-b135-da77275d4641 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:48.477526 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3373ad-4fef-4d51-b135-da77275d4641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:48.544247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8a3373ad-4fef-4d51-b135-da77275d4641 None None] MFA Rules not processed for user `e20c26710df6481288e8746a74410708`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:48.546364 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8a3373ad-4fef-4d51-b135-da77275d4641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:48.677970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1155/2287] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:48.709938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52664e5-b069-4b4f-9165-806e6f3be472 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:48.712679 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-d52664e5-b069-4b4f-9165-806e6f3be472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:48.801166 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52664e5-b069-4b4f-9165-806e6f3be472 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:48.813914 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d0896-0ac6-4be2-9723-43026bf19e8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:48.815180 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d0896-0ac6-4be2-9723-43026bf19e8f service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:48.815614 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d0896-0ac6-4be2-9723-43026bf19e8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:48.815907 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d0896-0ac6-4be2-9723-43026bf19e8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:48.825829 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d0896-0ac6-4be2-9723-43026bf19e8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:48.828506 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d0896-0ac6-4be2-9723-43026bf19e8f service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:48.853668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1133/2288] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:52.581336 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd8ce6-ab37-4b9a-99c7-bbbf534bb2ac None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:52.582228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd8ce6-ab37-4b9a-99c7-bbbf534bb2ac None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:52.582702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd8ce6-ab37-4b9a-99c7-bbbf534bb2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:52.653066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-fddd8ce6-ab37-4b9a-99c7-bbbf534bb2ac None None] MFA Rules not processed for user `3e06d1b0e1a9472db764f4be7e42fdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:52.657966 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fddd8ce6-ab37-4b9a-99c7-bbbf534bb2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:52.760062 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1156/2289] 10.222.0.76 () {58 vars in 1009 bytes} [Thu May 26 18:06:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:06:52.832740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d0896-0ac6-4be2-9723-43026bf19e8f service cinder] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:52.834232 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-375d0896-0ac6-4be2-9723-43026bf19e8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:52.927607 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70060243-7a5f-47ec-b831-3f8943ac9107 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:52.928084 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70060243-7a5f-47ec-b831-3f8943ac9107 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:52.928438 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70060243-7a5f-47ec-b831-3f8943ac9107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:52.986140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-70060243-7a5f-47ec-b831-3f8943ac9107 None None] MFA Rules not processed for user `42c96a85400941818b658cca8cb56fec`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:52.988054 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-70060243-7a5f-47ec-b831-3f8943ac9107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:53.039444 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d0896-0ac6-4be2-9723-43026bf19e8f service cinder] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:53.055697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb509156-8ce6-4e97-8027-757de6363a26 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ca774e864ca44cfb187e38bc8a8f6ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:53.056501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb509156-8ce6-4e97-8027-757de6363a26 service swift] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:53.056787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb509156-8ce6-4e97-8027-757de6363a26 service swift] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:53.057108 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb509156-8ce6-4e97-8027-757de6363a26 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:53.070273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:53.075834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb509156-8ce6-4e97-8027-757de6363a26 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:53.078553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb509156-8ce6-4e97-8027-757de6363a26 service swift] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:53.087742 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1134/2290] 10.222.0.76 () {60 vars in 1384 bytes} [Thu May 26 18:06:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:06:53.089625 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14612664-4b72-4ef2-b633-97849c2305db None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:53.089921 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14612664-4b72-4ef2-b633-97849c2305db None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:53.090318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14612664-4b72-4ef2-b633-97849c2305db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:53.100466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1157/2291] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:53.105760 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70060243-7a5f-47ec-b831-3f8943ac9107 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:53.107278 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-70060243-7a5f-47ec-b831-3f8943ac9107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:53.122780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-14612664-4b72-4ef2-b633-97849c2305db None None] MFA Rules not processed for user `dfbe23f3a0294148a88f4758d8470665`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:53.124839 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-14612664-4b72-4ef2-b633-97849c2305db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:53.217935 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70060243-7a5f-47ec-b831-3f8943ac9107 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:53.224422 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993291d5-82b6-42b1-b716-f4161364042d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:53.225193 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993291d5-82b6-42b1-b716-f4161364042d admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:53.225547 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993291d5-82b6-42b1-b716-f4161364042d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:53.225973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993291d5-82b6-42b1-b716-f4161364042d admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:53.226989 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993291d5-82b6-42b1-b716-f4161364042d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:53.229039 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993291d5-82b6-42b1-b716-f4161364042d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:53.236477 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1135/2292] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:53.281579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1158/2293] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:06:53 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:53.284740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14612664-4b72-4ef2-b633-97849c2305db None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:53.297065 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14612664-4b72-4ef2-b633-97849c2305db None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:53.339640 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:53.396697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b4e6a-c883-403d-8c44-d003b6d27f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:53.397493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b4e6a-c883-403d-8c44-d003b6d27f96 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:53.397847 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b4e6a-c883-403d-8c44-d003b6d27f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:53.398215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b4e6a-c883-403d-8c44-d003b6d27f96 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:53.399520 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b4e6a-c883-403d-8c44-d003b6d27f96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:53.402835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b4e6a-c883-403d-8c44-d003b6d27f96 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:53.433958 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1136/2294] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:06:53 2022] POST /identity/v3/users => generated 296 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:06:53.438650 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993291d5-82b6-42b1-b716-f4161364042d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:53.450539 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993291d5-82b6-42b1-b716-f4161364042d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:53.456715 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f5fd1-03da-4d19-88c7-4442ede775c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:53.458321 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f5fd1-03da-4d19-88c7-4442ede775c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:53.458321 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f5fd1-03da-4d19-88c7-4442ede775c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:53.458732 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f5fd1-03da-4d19-88c7-4442ede775c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:53.459476 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f5fd1-03da-4d19-88c7-4442ede775c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:53.461181 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f5fd1-03da-4d19-88c7-4442ede775c0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:53.492529 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1159/2295] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:06:53.496619 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b4e6a-c883-403d-8c44-d003b6d27f96 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:53.519934 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b4e6a-c883-403d-8c44-d003b6d27f96 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:53.533778 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db2cdc-83c4-413d-be2f-b96cf13b1c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:53.534843 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db2cdc-83c4-413d-be2f-b96cf13b1c21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:53.535193 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db2cdc-83c4-413d-be2f-b96cf13b1c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:53.535482 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db2cdc-83c4-413d-be2f-b96cf13b1c21 admin admin] PATH_INFO: `/v3/projects/d380d672826643ba9d6f491c3a981dff/users/9b89b641099e4132817893abbab6bf99/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:53.560817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db2cdc-83c4-413d-be2f-b96cf13b1c21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d380d672826643ba9d6f491c3a981dff, user_id=9b89b641099e4132817893abbab6bf99, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:53.565550 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db2cdc-83c4-413d-be2f-b96cf13b1c21 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:53.585712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1137/2296] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:53 2022] PUT /identity/v3/projects/d380d672826643ba9d6f491c3a981dff/users/9b89b641099e4132817893abbab6bf99/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:53.590092 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f5fd1-03da-4d19-88c7-4442ede775c0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:53.605935 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f5fd1-03da-4d19-88c7-4442ede775c0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:53.703086 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e106a-7900-4f42-a44c-9005f8915b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:53.704682 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e106a-7900-4f42-a44c-9005f8915b2b admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:53.705046 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e106a-7900-4f42-a44c-9005f8915b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:53.705327 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e106a-7900-4f42-a44c-9005f8915b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:53.706070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e106a-7900-4f42-a44c-9005f8915b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:53.707739 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e106a-7900-4f42-a44c-9005f8915b2b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:53.739244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1160/2297] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:06:53.743885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db2cdc-83c4-413d-be2f-b96cf13b1c21 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:53.758883 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db2cdc-83c4-413d-be2f-b96cf13b1c21 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:53.765358 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcecd03-3faa-4804-9831-c16cef2ff003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:53.766071 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcecd03-3faa-4804-9831-c16cef2ff003 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:53.766417 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcecd03-3faa-4804-9831-c16cef2ff003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:53.766698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcecd03-3faa-4804-9831-c16cef2ff003 admin admin] PATH_INFO: `/v3/projects/d380d672826643ba9d6f491c3a981dff/users/9b89b641099e4132817893abbab6bf99/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:53.768834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcecd03-3faa-4804-9831-c16cef2ff003 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d380d672826643ba9d6f491c3a981dff, user_id=9b89b641099e4132817893abbab6bf99, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:53.775551 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcecd03-3faa-4804-9831-c16cef2ff003 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:53.788209 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1138/2298] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:06:53 2022] PUT /identity/v3/projects/d380d672826643ba9d6f491c3a981dff/users/9b89b641099e4132817893abbab6bf99/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:53.900758 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5202197-03ba-4db2-a2d8-40a71f7f5fe6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:53.901184 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5202197-03ba-4db2-a2d8-40a71f7f5fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:53.901681 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5202197-03ba-4db2-a2d8-40a71f7f5fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:53.947653 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b5202197-03ba-4db2-a2d8-40a71f7f5fe6 None None] MFA Rules not processed for user `3e06d1b0e1a9472db764f4be7e42fdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:53.947653 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b5202197-03ba-4db2-a2d8-40a71f7f5fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:54.045321 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:54.067540 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1161/2299] 10.222.0.76 () {58 vars in 967 bytes} [Thu May 26 18:06:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:54.082831 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcecd03-3faa-4804-9831-c16cef2ff003 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:54.247039 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcecd03-3faa-4804-9831-c16cef2ff003 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:54.257199 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e175a6b-7614-475b-bc85-9674c37d4d76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ca774e864ca44cfb187e38bc8a8f6ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:54.258326 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e175a6b-7614-475b-bc85-9674c37d4d76 service swift] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:54.258950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e175a6b-7614-475b-bc85-9674c37d4d76 service swift] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:54.260171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e175a6b-7614-475b-bc85-9674c37d4d76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:54.283284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e175a6b-7614-475b-bc85-9674c37d4d76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:54.285142 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e175a6b-7614-475b-bc85-9674c37d4d76 service swift] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:54.297272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1139/2300] 10.222.0.76 () {60 vars in 1384 bytes} [Thu May 26 18:06:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:06:54.728234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7dc235-26e3-4bf1-9284-fe14a6c4071d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:54.728795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7dc235-26e3-4bf1-9284-fe14a6c4071d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:54.728795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7dc235-26e3-4bf1-9284-fe14a6c4071d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:54.773810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8f7dc235-26e3-4bf1-9284-fe14a6c4071d None None] MFA Rules not processed for user `3e06d1b0e1a9472db764f4be7e42fdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:54.776078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8f7dc235-26e3-4bf1-9284-fe14a6c4071d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:54.792249 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1162/2301] 10.222.0.76 () {58 vars in 1009 bytes} [Thu May 26 18:06:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:54.812592 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e175a6b-7614-475b-bc85-9674c37d4d76 service swift] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:54.833414 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e175a6b-7614-475b-bc85-9674c37d4d76 service swift] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:54.845394 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d09dd9-df49-4712-aeb5-238603577353 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ca774e864ca44cfb187e38bc8a8f6ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:54.846591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d09dd9-df49-4712-aeb5-238603577353 service swift] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:54.847404 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d09dd9-df49-4712-aeb5-238603577353 service swift] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:54.847892 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d09dd9-df49-4712-aeb5-238603577353 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:54.865882 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d09dd9-df49-4712-aeb5-238603577353 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:54.868896 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d09dd9-df49-4712-aeb5-238603577353 service swift] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:54.884342 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1140/2302] 10.222.0.76 () {60 vars in 1384 bytes} [Thu May 26 18:06:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 18:06:55.782800 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b357b7f-93d3-4988-b9c2-3c5dad5a47b0 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:55.783408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b357b7f-93d3-4988-b9c2-3c5dad5a47b0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:55.783408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b357b7f-93d3-4988-b9c2-3c5dad5a47b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:55.816718 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1b357b7f-93d3-4988-b9c2-3c5dad5a47b0 None None] MFA Rules not processed for user `3e06d1b0e1a9472db764f4be7e42fdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:55.817997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1b357b7f-93d3-4988-b9c2-3c5dad5a47b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:55.832355 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1163/2303] 10.222.0.76 () {58 vars in 967 bytes} [Thu May 26 18:06:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:55.849679 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d09dd9-df49-4712-aeb5-238603577353 service swift] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:55.869492 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d09dd9-df49-4712-aeb5-238603577353 service swift] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:55.894263 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82fd35-b237-46ea-b19c-398daad75f48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ca774e864ca44cfb187e38bc8a8f6ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:55.895319 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82fd35-b237-46ea-b19c-398daad75f48 service swift] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:55.895710 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82fd35-b237-46ea-b19c-398daad75f48 service swift] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:55.896026 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82fd35-b237-46ea-b19c-398daad75f48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:55.918908 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a82fd35-b237-46ea-b19c-398daad75f48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:55.927731 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a82fd35-b237-46ea-b19c-398daad75f48 service swift] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:55.955648 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1141/2304] 10.222.0.76 () {60 vars in 1384 bytes} [Thu May 26 18:06:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:06:58.578938 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b357b7f-93d3-4988-b9c2-3c5dad5a47b0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:58.673157 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b357b7f-93d3-4988-b9c2-3c5dad5a47b0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:58.677934 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098306e-8d6d-41eb-9edc-6085fee5618e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:58.678869 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098306e-8d6d-41eb-9edc-6085fee5618e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:58.679159 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098306e-8d6d-41eb-9edc-6085fee5618e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:58.679439 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098306e-8d6d-41eb-9edc-6085fee5618e admin admin] PATH_INFO: `/v3/users/e20c26710df6481288e8746a74410708` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:58.680947 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6098306e-8d6d-41eb-9edc-6085fee5618e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e20c26710df6481288e8746a74410708)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:58.682974 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6098306e-8d6d-41eb-9edc-6085fee5618e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:58.781502 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6098306e-8d6d-41eb-9edc-6085fee5618e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e20c26710df6481288e8746a74410708'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:58.792330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6098306e-8d6d-41eb-9edc-6085fee5618e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e20c26710df6481288e8746a74410708'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:58.826571 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6098306e-8d6d-41eb-9edc-6085fee5618e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e20c26710df6481288e8746a74410708'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:58.827684 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6098306e-8d6d-41eb-9edc-6085fee5618e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e20c26710df6481288e8746a74410708'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:58.828759 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6098306e-8d6d-41eb-9edc-6085fee5618e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e20c26710df6481288e8746a74410708'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:58.861512 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1164/2305] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:58 2022] DELETE /identity/v3/users/e20c26710df6481288e8746a74410708 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:06:58.867283 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a6096-18e9-4b77-b15f-4f27acbdbf25 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:58.868180 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a6096-18e9-4b77-b15f-4f27acbdbf25 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:58.868621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a6096-18e9-4b77-b15f-4f27acbdbf25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:58.921399 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098306e-8d6d-41eb-9edc-6085fee5618e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:58.924024 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6098306e-8d6d-41eb-9edc-6085fee5618e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:58.930361 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-be1a6096-18e9-4b77-b15f-4f27acbdbf25 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:58.932376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-be1a6096-18e9-4b77-b15f-4f27acbdbf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:59.124102 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:59.169277 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098306e-8d6d-41eb-9edc-6085fee5618e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:59.183804 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb3c7d-eb6c-4e3c-8e1b-c80745d83c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:59.184941 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb3c7d-eb6c-4e3c-8e1b-c80745d83c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:59.185329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb3c7d-eb6c-4e3c-8e1b-c80745d83c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:59.185815 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb3c7d-eb6c-4e3c-8e1b-c80745d83c51 admin admin] PATH_INFO: `/v3/users/d9790a81d03741688e45948876714cb4` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:59.189146 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb3c7d-eb6c-4e3c-8e1b-c80745d83c51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9790a81d03741688e45948876714cb4)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:59.194940 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb3c7d-eb6c-4e3c-8e1b-c80745d83c51 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:59.225734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:06:59.231133 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1142/2306] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:59.330568 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a6096-18e9-4b77-b15f-4f27acbdbf25 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:59.333945 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-be1a6096-18e9-4b77-b15f-4f27acbdbf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:59.386329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c0fb3c7d-eb6c-4e3c-8e1b-c80745d83c51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9790a81d03741688e45948876714cb4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:59.401954 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c0fb3c7d-eb6c-4e3c-8e1b-c80745d83c51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9790a81d03741688e45948876714cb4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:59.437064 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c0fb3c7d-eb6c-4e3c-8e1b-c80745d83c51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9790a81d03741688e45948876714cb4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:59.442461 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c0fb3c7d-eb6c-4e3c-8e1b-c80745d83c51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9790a81d03741688e45948876714cb4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:59.444338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c0fb3c7d-eb6c-4e3c-8e1b-c80745d83c51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9790a81d03741688e45948876714cb4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:06:59.481890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1165/2307] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:06:58 2022] DELETE /identity/v3/users/d9790a81d03741688e45948876714cb4 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:06:59.488892 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b398e-85e4-4df9-8828-101aaf54a532 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:59.489191 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b398e-85e4-4df9-8828-101aaf54a532 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:59.489497 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b398e-85e4-4df9-8828-101aaf54a532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:59.514720 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-883b398e-85e4-4df9-8828-101aaf54a532 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:06:59.517780 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-883b398e-85e4-4df9-8828-101aaf54a532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:59.595461 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a6096-18e9-4b77-b15f-4f27acbdbf25 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:59.671801 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1166/2308] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:06:59.678509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b398e-85e4-4df9-8828-101aaf54a532 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:06:59.680102 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-883b398e-85e4-4df9-8828-101aaf54a532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:59.705575 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758a70b-f6b5-4bba-a742-10b721daa819 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:59.707815 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758a70b-f6b5-4bba-a742-10b721daa819 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:59.708716 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758a70b-f6b5-4bba-a742-10b721daa819 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:59.709453 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758a70b-f6b5-4bba-a742-10b721daa819 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:59.712122 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-7758a70b-f6b5-4bba-a742-10b721daa819 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:06:59.726716 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758a70b-f6b5-4bba-a742-10b721daa819 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:59.728923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758a70b-f6b5-4bba-a742-10b721daa819 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:59.740844 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1143/2309] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:06:59.793626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b398e-85e4-4df9-8828-101aaf54a532 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:06:59.816538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0526e-5d8d-40ef-a455-d0b5c844dda1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:06:59.818333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0526e-5d8d-40ef-a455-d0b5c844dda1 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:06:59.820766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0526e-5d8d-40ef-a455-d0b5c844dda1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:06:59.821397 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0526e-5d8d-40ef-a455-d0b5c844dda1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:06:59.841697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a0526e-5d8d-40ef-a455-d0b5c844dda1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:06:59.843765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a0526e-5d8d-40ef-a455-d0b5c844dda1 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:06:59.857282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1167/2310] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:06:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:07:00.651781 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758a70b-f6b5-4bba-a742-10b721daa819 service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:00.655027 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-7758a70b-f6b5-4bba-a742-10b721daa819 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:00.687144 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0526e-5d8d-40ef-a455-d0b5c844dda1 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:00.687609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758a70b-f6b5-4bba-a742-10b721daa819 service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:00.689895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a3a0526e-5d8d-40ef-a455-d0b5c844dda1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:00.708393 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef96ea-e0ff-4e44-9be8-f9b08ee94e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:00.709333 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef96ea-e0ff-4e44-9be8-f9b08ee94e74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:00.709623 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef96ea-e0ff-4e44-9be8-f9b08ee94e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:00.709905 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef96ea-e0ff-4e44-9be8-f9b08ee94e74 admin admin] PATH_INFO: `/v3/projects/799f6c254b56426e89102354e6274b18` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:00.721684 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef96ea-e0ff-4e44-9be8-f9b08ee94e74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=799f6c254b56426e89102354e6274b18)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:00.725612 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef96ea-e0ff-4e44-9be8-f9b08ee94e74 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:00.770379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0526e-5d8d-40ef-a455-d0b5c844dda1 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:00.783742 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b1231f-7b67-4f72-b7dc-16c440427cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:00.790906 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1231f-7b67-4f72-b7dc-16c440427cd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:00.792604 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1231f-7b67-4f72-b7dc-16c440427cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:00.794318 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1231f-7b67-4f72-b7dc-16c440427cd1 admin admin] PATH_INFO: `/v3/projects/cf1c9dce6a9c48aeaff0d0e503f72078` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:00.798609 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b1231f-7b67-4f72-b7dc-16c440427cd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf1c9dce6a9c48aeaff0d0e503f72078)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:00.801112 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b1231f-7b67-4f72-b7dc-16c440427cd1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:00.854272 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:00.858132 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-60ef96ea-e0ff-4e44-9be8-f9b08ee94e74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '799f6c254b56426e89102354e6274b18', 'request_id': 'req-60ef96ea-e0ff-4e44-9be8-f9b08ee94e74'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:00.877622 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-60ef96ea-e0ff-4e44-9be8-f9b08ee94e74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '799f6c254b56426e89102354e6274b18', 'request_id': 'req-60ef96ea-e0ff-4e44-9be8-f9b08ee94e74'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:00.905407 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-60ef96ea-e0ff-4e44-9be8-f9b08ee94e74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '799f6c254b56426e89102354e6274b18', 'request_id': 'req-60ef96ea-e0ff-4e44-9be8-f9b08ee94e74'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:00.942056 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-54b1231f-7b67-4f72-b7dc-16c440427cd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf1c9dce6a9c48aeaff0d0e503f72078', 'request_id': 'req-54b1231f-7b67-4f72-b7dc-16c440427cd1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:00.957379 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-60ef96ea-e0ff-4e44-9be8-f9b08ee94e74 admin admin] The token cache is being invalidate because project 799f6c254b56426e89102354e6274b18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:07:00.957677 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-60ef96ea-e0ff-4e44-9be8-f9b08ee94e74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:00.958478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-60ef96ea-e0ff-4e44-9be8-f9b08ee94e74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:00.962659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-54b1231f-7b67-4f72-b7dc-16c440427cd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf1c9dce6a9c48aeaff0d0e503f72078', 'request_id': 'req-54b1231f-7b67-4f72-b7dc-16c440427cd1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:00.970552 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1144/2311] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:07:00 2022] DELETE /identity/v3/projects/799f6c254b56426e89102354e6274b18 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:00.972296 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80640396-1088-4818-aeeb-c4a157f36d70 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:00.975515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80640396-1088-4818-aeeb-c4a157f36d70 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:00.976316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80640396-1088-4818-aeeb-c4a157f36d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:00.978428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-54b1231f-7b67-4f72-b7dc-16c440427cd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf1c9dce6a9c48aeaff0d0e503f72078', 'request_id': 'req-54b1231f-7b67-4f72-b7dc-16c440427cd1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:01.031968 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-54b1231f-7b67-4f72-b7dc-16c440427cd1 admin admin] The token cache is being invalidate because project cf1c9dce6a9c48aeaff0d0e503f72078 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:07:01.032533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-54b1231f-7b67-4f72-b7dc-16c440427cd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:01.033665 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-54b1231f-7b67-4f72-b7dc-16c440427cd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:01.038849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1168/2312] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:07:00 2022] DELETE /identity/v3/projects/cf1c9dce6a9c48aeaff0d0e503f72078 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:07:01.039757 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-80640396-1088-4818-aeeb-c4a157f36d70 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:01.041184 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-80640396-1088-4818-aeeb-c4a157f36d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:01.046149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff807c74-d1ca-4bea-bf2b-1d4f155dad73 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:01.046496 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff807c74-d1ca-4bea-bf2b-1d4f155dad73 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:01.046778 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff807c74-d1ca-4bea-bf2b-1d4f155dad73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:01.099262 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:01.138334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ff807c74-d1ca-4bea-bf2b-1d4f155dad73 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:01.138334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ff807c74-d1ca-4bea-bf2b-1d4f155dad73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:01.220278 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1145/2313] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:01.222237 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0515f-a163-4987-af6a-ee9c3de56ab4 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:01.222657 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0515f-a163-4987-af6a-ee9c3de56ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:01.223028 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0515f-a163-4987-af6a-ee9c3de56ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:01.267526 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-5ac0515f-a163-4987-af6a-ee9c3de56ab4 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:01.267739 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-5ac0515f-a163-4987-af6a-ee9c3de56ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:01.298008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1146/2314] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:01.298932 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1169/2315] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:01.303375 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0515f-a163-4987-af6a-ee9c3de56ab4 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:01.307427 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5f985-0dd4-4b7f-b4f3-b275c398eac6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:01.307719 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5f985-0dd4-4b7f-b4f3-b275c398eac6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:01.308001 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5f985-0dd4-4b7f-b4f3-b275c398eac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:01.340506 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0515f-a163-4987-af6a-ee9c3de56ab4 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:01.351313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-fea5f985-0dd4-4b7f-b4f3-b275c398eac6 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:01.352509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fea5f985-0dd4-4b7f-b4f3-b275c398eac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:01.359836 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57633c86-8824-4db3-bfcc-d82cfd5bb849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:01.360542 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57633c86-8824-4db3-bfcc-d82cfd5bb849 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:01.360830 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57633c86-8824-4db3-bfcc-d82cfd5bb849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:01.361108 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57633c86-8824-4db3-bfcc-d82cfd5bb849 admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:01.361864 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57633c86-8824-4db3-bfcc-d82cfd5bb849 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:01.362172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57633c86-8824-4db3-bfcc-d82cfd5bb849 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:01.363921 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57633c86-8824-4db3-bfcc-d82cfd5bb849 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:01.391392 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1170/2316] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:01.394451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5f985-0dd4-4b7f-b4f3-b275c398eac6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:01.415065 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1147/2317] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:01.416185 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57633c86-8824-4db3-bfcc-d82cfd5bb849 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:01.420108 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5f985-0dd4-4b7f-b4f3-b275c398eac6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:01.437724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57633c86-8824-4db3-bfcc-d82cfd5bb849 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:01.449814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e34fc-6c2b-4940-bc4d-1efd3ac9c3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:01.450679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e34fc-6c2b-4940-bc4d-1efd3ac9c3a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:01.451102 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e34fc-6c2b-4940-bc4d-1efd3ac9c3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:01.452611 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e34fc-6c2b-4940-bc4d-1efd3ac9c3a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:01.455090 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e34fc-6c2b-4940-bc4d-1efd3ac9c3a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:01.455606 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e34fc-6c2b-4940-bc4d-1efd3ac9c3a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:01.457775 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fcd89-bd5b-4cc0-bd9e-e843008f7bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:01.458510 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fcd89-bd5b-4cc0-bd9e-e843008f7bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:01.458793 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fcd89-bd5b-4cc0-bd9e-e843008f7bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:01.459291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fcd89-bd5b-4cc0-bd9e-e843008f7bf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:01.459764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e34fc-6c2b-4940-bc4d-1efd3ac9c3a1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:01.459888 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4fcd89-bd5b-4cc0-bd9e-e843008f7bf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:01.461703 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4fcd89-bd5b-4cc0-bd9e-e843008f7bf4 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:01.493024 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1171/2318] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:01.494725 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e34fc-6c2b-4940-bc4d-1efd3ac9c3a1 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:01.521225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1148/2319] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:07:01 2022] POST /identity/v3/projects => generated 391 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:01.525464 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fcd89-bd5b-4cc0-bd9e-e843008f7bf4 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:01.534993 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e34fc-6c2b-4940-bc4d-1efd3ac9c3a1 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:01.665058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fcd89-bd5b-4cc0-bd9e-e843008f7bf4 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:01.672562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adde57b-3775-4c01-ae01-898c8397279b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:01.673641 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adde57b-3775-4c01-ae01-898c8397279b admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:01.674316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adde57b-3775-4c01-ae01-898c8397279b admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:01.674867 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adde57b-3775-4c01-ae01-898c8397279b admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:01.675783 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adde57b-3775-4c01-ae01-898c8397279b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:01.679809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adde57b-3775-4c01-ae01-898c8397279b admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:01.689609 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b4dc0-f438-4b9e-a0e9-6cb119fb5a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:01.690649 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b4dc0-f438-4b9e-a0e9-6cb119fb5a69 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:01.691236 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b4dc0-f438-4b9e-a0e9-6cb119fb5a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:01.691822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b4dc0-f438-4b9e-a0e9-6cb119fb5a69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:01.692687 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b4dc0-f438-4b9e-a0e9-6cb119fb5a69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:01.694842 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b4dc0-f438-4b9e-a0e9-6cb119fb5a69 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:01.750643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1149/2320] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:07:01 2022] POST /identity/v3/users => generated 296 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:07:01.756216 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adde57b-3775-4c01-ae01-898c8397279b admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:01.777272 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1172/2321] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:07:01 2022] POST /identity/v3/projects => generated 373 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:01.790723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adde57b-3775-4c01-ae01-898c8397279b admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:01.791399 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b4dc0-f438-4b9e-a0e9-6cb119fb5a69 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:01.842702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:01.952329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28d818-ae5f-4ffd-91b1-12034e805f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:01.953027 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28d818-ae5f-4ffd-91b1-12034e805f75 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:01.953316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28d818-ae5f-4ffd-91b1-12034e805f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:01.953867 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28d818-ae5f-4ffd-91b1-12034e805f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:01.954710 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28d818-ae5f-4ffd-91b1-12034e805f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:01.956403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28d818-ae5f-4ffd-91b1-12034e805f75 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:01.965581 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b4dc0-f438-4b9e-a0e9-6cb119fb5a69 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:01.985123 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1150/2322] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:01.995788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086cb135-8ed8-4a58-8ee0-8ba3cea424ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:01.996543 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086cb135-8ed8-4a58-8ee0-8ba3cea424ee admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:01.996850 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086cb135-8ed8-4a58-8ee0-8ba3cea424ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:01.997133 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086cb135-8ed8-4a58-8ee0-8ba3cea424ee admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:01.997871 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086cb135-8ed8-4a58-8ee0-8ba3cea424ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:02.001689 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28d818-ae5f-4ffd-91b1-12034e805f75 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:02.008594 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086cb135-8ed8-4a58-8ee0-8ba3cea424ee admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:02.027092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28d818-ae5f-4ffd-91b1-12034e805f75 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:02.038834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae7f15b-a7d3-4f98-b7f5-ca5da8674abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:02.040315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7f15b-a7d3-4f98-b7f5-ca5da8674abe admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:02.040609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7f15b-a7d3-4f98-b7f5-ca5da8674abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:02.040895 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7f15b-a7d3-4f98-b7f5-ca5da8674abe admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:02.041648 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae7f15b-a7d3-4f98-b7f5-ca5da8674abe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:02.047130 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae7f15b-a7d3-4f98-b7f5-ca5da8674abe admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:02.054289 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1173/2323] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:07:01 2022] POST /identity/v3/users => generated 287 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:02.073000 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086cb135-8ed8-4a58-8ee0-8ba3cea424ee admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:02.081746 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:02.119932 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086cb135-8ed8-4a58-8ee0-8ba3cea424ee admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:02.153620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1151/2324] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:02.156130 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae7f15b-a7d3-4f98-b7f5-ca5da8674abe admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:02.214574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ea42c-f829-45fb-b657-ffae5fe690e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:02.216229 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ea42c-f829-45fb-b657-ffae5fe690e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:02.217319 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ea42c-f829-45fb-b657-ffae5fe690e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:02.217867 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ea42c-f829-45fb-b657-ffae5fe690e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:02.219031 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ea42c-f829-45fb-b657-ffae5fe690e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:02.223498 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ea42c-f829-45fb-b657-ffae5fe690e4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:02.229803 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae7f15b-a7d3-4f98-b7f5-ca5da8674abe admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:02.245799 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18073cd5-ef71-41d7-9bbf-f3eb6388aea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:02.249424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18073cd5-ef71-41d7-9bbf-f3eb6388aea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:02.251952 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18073cd5-ef71-41d7-9bbf-f3eb6388aea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:02.253096 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18073cd5-ef71-41d7-9bbf-f3eb6388aea2 admin admin] PATH_INFO: `/v3/projects/8a31bff554fd4c6f8f84fa87f14ec8c8/users/eb9fa9855586408f9c50b3d7999965dc/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:02.254533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1174/2325] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:02.257054 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ea42c-f829-45fb-b657-ffae5fe690e4 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:02.278491 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ea42c-f829-45fb-b657-ffae5fe690e4 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:02.297400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25deed81-60c2-4453-834d-708f8463d447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:02.298927 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18073cd5-ef71-41d7-9bbf-f3eb6388aea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a31bff554fd4c6f8f84fa87f14ec8c8, user_id=eb9fa9855586408f9c50b3d7999965dc, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:02.299863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25deed81-60c2-4453-834d-708f8463d447 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:02.302714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25deed81-60c2-4453-834d-708f8463d447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:02.303200 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25deed81-60c2-4453-834d-708f8463d447 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:02.306829 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25deed81-60c2-4453-834d-708f8463d447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:02.307894 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18073cd5-ef71-41d7-9bbf-f3eb6388aea2 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:02.309510 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25deed81-60c2-4453-834d-708f8463d447 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:02.332301 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1175/2326] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:02.337175 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25deed81-60c2-4453-834d-708f8463d447 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:02.352598 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1152/2327] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:07:02 2022] PUT /identity/v3/projects/8a31bff554fd4c6f8f84fa87f14ec8c8/users/eb9fa9855586408f9c50b3d7999965dc/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:02.363609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0e082-1af1-4758-b93d-3e1243c5506b None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:02.363835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0e082-1af1-4758-b93d-3e1243c5506b None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:02.364109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0e082-1af1-4758-b93d-3e1243c5506b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:02.370863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25deed81-60c2-4453-834d-708f8463d447 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:02.443192 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-b2b0e082-1af1-4758-b93d-3e1243c5506b None None] MFA Rules not processed for user `eb9fa9855586408f9c50b3d7999965dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:02.444480 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-b2b0e082-1af1-4758-b93d-3e1243c5506b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:02.546369 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a909e-004b-41d4-b7ba-5d2bd6ca7688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:02.547502 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a909e-004b-41d4-b7ba-5d2bd6ca7688 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:02.549439 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a909e-004b-41d4-b7ba-5d2bd6ca7688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:02.549822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a909e-004b-41d4-b7ba-5d2bd6ca7688 admin admin] PATH_INFO: `/v3/projects/c2cf8fb59b4f4b12900d336f1d91b1eb/users/b2ba168683a4433f8a763cb4c250c9a0/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:02.590753 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a909e-004b-41d4-b7ba-5d2bd6ca7688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2cf8fb59b4f4b12900d336f1d91b1eb, user_id=b2ba168683a4433f8a763cb4c250c9a0, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:02.594000 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a909e-004b-41d4-b7ba-5d2bd6ca7688 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:02.603881 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1153/2328] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:02.617522 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b0e082-1af1-4758-b93d-3e1243c5506b None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:02.619085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-b2b0e082-1af1-4758-b93d-3e1243c5506b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:02.627927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1176/2329] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:07:02 2022] PUT /identity/v3/projects/c2cf8fb59b4f4b12900d336f1d91b1eb/users/b2ba168683a4433f8a763cb4c250c9a0/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:02.632518 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901d5da-d817-42db-bc2f-63fa8e188778 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:02.632811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901d5da-d817-42db-bc2f-63fa8e188778 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:02.633094 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901d5da-d817-42db-bc2f-63fa8e188778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:02.658478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4901d5da-d817-42db-bc2f-63fa8e188778 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:02.661166 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4901d5da-d817-42db-bc2f-63fa8e188778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:02.733937 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b0e082-1af1-4758-b93d-3e1243c5506b None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:02.771108 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1177/2330] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:07:02.781743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901d5da-d817-42db-bc2f-63fa8e188778 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:02.783521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4901d5da-d817-42db-bc2f-63fa8e188778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:02.794162 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:02.804123 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a999a98-9f83-4844-8053-e2dc7233aa60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:02.804907 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a999a98-9f83-4844-8053-e2dc7233aa60 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:02.805268 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a999a98-9f83-4844-8053-e2dc7233aa60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:02.805619 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a999a98-9f83-4844-8053-e2dc7233aa60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:02.833536 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:02.859682 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901d5da-d817-42db-bc2f-63fa8e188778 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:02.864160 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57fddd-0e01-4829-9064-0a15452e7142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:02.866012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57fddd-0e01-4829-9064-0a15452e7142 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:02.866012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57fddd-0e01-4829-9064-0a15452e7142 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:02.866012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57fddd-0e01-4829-9064-0a15452e7142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:02.877380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57fddd-0e01-4829-9064-0a15452e7142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:02.879236 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57fddd-0e01-4829-9064-0a15452e7142 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:02.891788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1178/2331] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:07:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:07:02.918645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a999a98-9f83-4844-8053-e2dc7233aa60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:02.920631 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a999a98-9f83-4844-8053-e2dc7233aa60 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:02.934153 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1154/2332] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:07:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:05.032088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6765a-895f-4802-8db6-ad1b7acb7ca8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:05.032813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6765a-895f-4802-8db6-ad1b7acb7ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:05.033092 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6765a-895f-4802-8db6-ad1b7acb7ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:05.120438 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-cae6765a-895f-4802-8db6-ad1b7acb7ca8 None None] MFA Rules not processed for user `9b89b641099e4132817893abbab6bf99`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:05.121534 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cae6765a-895f-4802-8db6-ad1b7acb7ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:05.313408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1179/2333] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:05 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:05.333047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a999a98-9f83-4844-8053-e2dc7233aa60 service cinder] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:05.343469 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-6a999a98-9f83-4844-8053-e2dc7233aa60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:05.544795 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a999a98-9f83-4844-8053-e2dc7233aa60 service cinder] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:05.551893 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fccfc0-3a80-4593-9d3f-5a252da74b1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:05.553439 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fccfc0-3a80-4593-9d3f-5a252da74b1c service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:05.553876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fccfc0-3a80-4593-9d3f-5a252da74b1c service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:05.554360 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fccfc0-3a80-4593-9d3f-5a252da74b1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:05.556873 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-46fccfc0-3a80-4593-9d3f-5a252da74b1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:05.621259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fccfc0-3a80-4593-9d3f-5a252da74b1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:05.623708 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fccfc0-3a80-4593-9d3f-5a252da74b1c service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:05.633893 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1155/2334] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:07:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:08.811320 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6765a-895f-4802-8db6-ad1b7acb7ca8 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:08.814166 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cae6765a-895f-4802-8db6-ad1b7acb7ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:08.842947 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6765a-895f-4802-8db6-ad1b7acb7ca8 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:08.859393 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e970147-20d3-4c83-9d2d-1eabc942e3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:08.860587 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e970147-20d3-4c83-9d2d-1eabc942e3d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:08.862390 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e970147-20d3-4c83-9d2d-1eabc942e3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:08.863437 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e970147-20d3-4c83-9d2d-1eabc942e3d7 admin admin] PATH_INFO: `/v3/users/372e7144b26748fea5ad297493142cc2` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:08.869864 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e970147-20d3-4c83-9d2d-1eabc942e3d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=372e7144b26748fea5ad297493142cc2)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:08.872067 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e970147-20d3-4c83-9d2d-1eabc942e3d7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:09.020269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6e970147-20d3-4c83-9d2d-1eabc942e3d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '372e7144b26748fea5ad297493142cc2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:09.032717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6e970147-20d3-4c83-9d2d-1eabc942e3d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '372e7144b26748fea5ad297493142cc2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:09.061981 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6e970147-20d3-4c83-9d2d-1eabc942e3d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '372e7144b26748fea5ad297493142cc2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:09.064243 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6e970147-20d3-4c83-9d2d-1eabc942e3d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '372e7144b26748fea5ad297493142cc2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:09.067030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6e970147-20d3-4c83-9d2d-1eabc942e3d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '372e7144b26748fea5ad297493142cc2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:09.122546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1180/2335] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:07:08 2022] DELETE /identity/v3/users/372e7144b26748fea5ad297493142cc2 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:09.126995 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e385a4f-3d9c-4d9a-aaa8-662de49b7f9e None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:09.127493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e385a4f-3d9c-4d9a-aaa8-662de49b7f9e None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:09.127587 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e385a4f-3d9c-4d9a-aaa8-662de49b7f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:09.196153 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-4e385a4f-3d9c-4d9a-aaa8-662de49b7f9e None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:09.197385 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-4e385a4f-3d9c-4d9a-aaa8-662de49b7f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:09.269169 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:09.356066 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1156/2336] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:09.379570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e970147-20d3-4c83-9d2d-1eabc942e3d7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:09.381585 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6e970147-20d3-4c83-9d2d-1eabc942e3d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:09.401409 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:09.512991 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e970147-20d3-4c83-9d2d-1eabc942e3d7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:09.531406 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb9016-3fc6-4a00-b5fb-76a282b33803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:09.533186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb9016-3fc6-4a00-b5fb-76a282b33803 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:09.533671 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb9016-3fc6-4a00-b5fb-76a282b33803 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:09.534552 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb9016-3fc6-4a00-b5fb-76a282b33803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:09.550584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb9016-3fc6-4a00-b5fb-76a282b33803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:09.564269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb9016-3fc6-4a00-b5fb-76a282b33803 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:09.593773 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1181/2337] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:07:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:10.450752 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e385a4f-3d9c-4d9a-aaa8-662de49b7f9e None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:10.454758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-4e385a4f-3d9c-4d9a-aaa8-662de49b7f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:10.476566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e385a4f-3d9c-4d9a-aaa8-662de49b7f9e None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:10.492998 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933c4f9-cc7e-44fd-b311-ad948afec7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:10.494224 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933c4f9-cc7e-44fd-b311-ad948afec7ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:10.495227 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933c4f9-cc7e-44fd-b311-ad948afec7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:10.495667 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933c4f9-cc7e-44fd-b311-ad948afec7ec admin admin] PATH_INFO: `/v3/projects/a08c291180e4486eaabb9c1e62905ab0` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:10.498682 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933c4f9-cc7e-44fd-b311-ad948afec7ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=a08c291180e4486eaabb9c1e62905ab0)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:10.501486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933c4f9-cc7e-44fd-b311-ad948afec7ec admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:10.613338 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-9933c4f9-cc7e-44fd-b311-ad948afec7ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a08c291180e4486eaabb9c1e62905ab0', 'request_id': 'req-9933c4f9-cc7e-44fd-b311-ad948afec7ec'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:10.627099 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-9933c4f9-cc7e-44fd-b311-ad948afec7ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a08c291180e4486eaabb9c1e62905ab0', 'request_id': 'req-9933c4f9-cc7e-44fd-b311-ad948afec7ec'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:10.641480 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-9933c4f9-cc7e-44fd-b311-ad948afec7ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a08c291180e4486eaabb9c1e62905ab0', 'request_id': 'req-9933c4f9-cc7e-44fd-b311-ad948afec7ec'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:10.688101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-9933c4f9-cc7e-44fd-b311-ad948afec7ec admin admin] The token cache is being invalidate because project a08c291180e4486eaabb9c1e62905ab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:07:10.688467 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-9933c4f9-cc7e-44fd-b311-ad948afec7ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:10.697835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-9933c4f9-cc7e-44fd-b311-ad948afec7ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:10.720008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1157/2338] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:07:10 2022] DELETE /identity/v3/projects/a08c291180e4486eaabb9c1e62905ab0 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:10.733251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ab0ae-8ad2-4048-ac9f-220f5489ad57 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:10.733709 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ab0ae-8ad2-4048-ac9f-220f5489ad57 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:10.733709 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ab0ae-8ad2-4048-ac9f-220f5489ad57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:10.761719 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ea7ab0ae-8ad2-4048-ac9f-220f5489ad57 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:10.763905 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ea7ab0ae-8ad2-4048-ac9f-220f5489ad57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:10.947816 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08edae2b-6aab-4fe3-803c-ec468c449131 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:10.948935 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1182/2339] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:10.949312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08edae2b-6aab-4fe3-803c-ec468c449131 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:10.949823 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08edae2b-6aab-4fe3-803c-ec468c449131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:10.985275 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-08edae2b-6aab-4fe3-803c-ec468c449131 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:10.987012 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-08edae2b-6aab-4fe3-803c-ec468c449131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:11.003087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1158/2340] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:11.015336 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ab0ae-8ad2-4048-ac9f-220f5489ad57 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:11.032951 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ab0ae-8ad2-4048-ac9f-220f5489ad57 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:11.043054 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952096d-f3ab-40b5-9e95-5f3618e2c4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:11.044790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952096d-f3ab-40b5-9e95-5f3618e2c4d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:11.045640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952096d-f3ab-40b5-9e95-5f3618e2c4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:11.046443 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952096d-f3ab-40b5-9e95-5f3618e2c4d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:11.048271 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952096d-f3ab-40b5-9e95-5f3618e2c4d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:11.048827 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952096d-f3ab-40b5-9e95-5f3618e2c4d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:11.052131 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952096d-f3ab-40b5-9e95-5f3618e2c4d0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:11.081149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1183/2341] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:11.085296 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08edae2b-6aab-4fe3-803c-ec468c449131 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:11.098590 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08edae2b-6aab-4fe3-803c-ec468c449131 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:11.104683 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303a642-6258-4f08-922d-ec3962fcadfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:11.105189 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303a642-6258-4f08-922d-ec3962fcadfb admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:11.112377 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303a642-6258-4f08-922d-ec3962fcadfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:11.112924 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303a642-6258-4f08-922d-ec3962fcadfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:11.113965 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303a642-6258-4f08-922d-ec3962fcadfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:11.116372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303a642-6258-4f08-922d-ec3962fcadfb admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:11.167436 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1159/2342] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:07:11 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:11.171957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952096d-f3ab-40b5-9e95-5f3618e2c4d0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:11.183253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952096d-f3ab-40b5-9e95-5f3618e2c4d0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:11.332285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa95e1-ff52-49e8-910a-21e940fc188a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:11.341413 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa95e1-ff52-49e8-910a-21e940fc188a admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:11.345340 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa95e1-ff52-49e8-910a-21e940fc188a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:11.346604 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa95e1-ff52-49e8-910a-21e940fc188a admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:11.348854 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa95e1-ff52-49e8-910a-21e940fc188a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:11.352946 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa95e1-ff52-49e8-910a-21e940fc188a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:11.421280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1184/2343] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:07:11 2022] POST /identity/v3/users => generated 287 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:07:11.435820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303a642-6258-4f08-922d-ec3962fcadfb admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:11.499057 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303a642-6258-4f08-922d-ec3962fcadfb admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:11.520867 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bfadd-ab2a-4050-875e-88e7ba4a2b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:11.522232 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bfadd-ab2a-4050-875e-88e7ba4a2b12 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:11.522729 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bfadd-ab2a-4050-875e-88e7ba4a2b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:11.523302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bfadd-ab2a-4050-875e-88e7ba4a2b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:11.524383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36bfadd-ab2a-4050-875e-88e7ba4a2b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:11.527205 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36bfadd-ab2a-4050-875e-88e7ba4a2b12 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:11.563450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1160/2344] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:11.565340 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa95e1-ff52-49e8-910a-21e940fc188a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:11.584353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa95e1-ff52-49e8-910a-21e940fc188a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:11.602487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f5d87-67d7-4433-bc91-20da3a0a6d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:11.603559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f5d87-67d7-4433-bc91-20da3a0a6d1d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:11.604001 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f5d87-67d7-4433-bc91-20da3a0a6d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:11.604945 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f5d87-67d7-4433-bc91-20da3a0a6d1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:11.605883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f5d87-67d7-4433-bc91-20da3a0a6d1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:11.608224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f5d87-67d7-4433-bc91-20da3a0a6d1d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:11.631301 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1185/2345] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:11.636291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bfadd-ab2a-4050-875e-88e7ba4a2b12 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:11.651725 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bfadd-ab2a-4050-875e-88e7ba4a2b12 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:11.663991 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d58f9d-58a5-42c2-9d91-21b8fcf6c41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:11.665025 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d58f9d-58a5-42c2-9d91-21b8fcf6c41e admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:11.665378 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d58f9d-58a5-42c2-9d91-21b8fcf6c41e admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:11.665820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d58f9d-58a5-42c2-9d91-21b8fcf6c41e admin admin] PATH_INFO: `/v3/projects/4bb46b1882464c04a476935ff315eb97/users/5158149d96fe42c795a630b301dbd53f/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:11.700920 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d58f9d-58a5-42c2-9d91-21b8fcf6c41e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bb46b1882464c04a476935ff315eb97, user_id=5158149d96fe42c795a630b301dbd53f, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:11.705342 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d58f9d-58a5-42c2-9d91-21b8fcf6c41e admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:11.730501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1161/2346] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:07:11 2022] PUT /identity/v3/projects/4bb46b1882464c04a476935ff315eb97/users/5158149d96fe42c795a630b301dbd53f/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:11.740049 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbe014-348b-4a21-82a6-62978e6b2234 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:11.740049 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbe014-348b-4a21-82a6-62978e6b2234 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:11.740829 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbe014-348b-4a21-82a6-62978e6b2234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:11.801061 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-90dbe014-348b-4a21-82a6-62978e6b2234 None None] MFA Rules not processed for user `5158149d96fe42c795a630b301dbd53f`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:11.802803 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-90dbe014-348b-4a21-82a6-62978e6b2234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:11.809820 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:11.822650 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e489d3c-3b81-4620-a140-85938b412a01 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:11.823166 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e489d3c-3b81-4620-a140-85938b412a01 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:11.823627 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e489d3c-3b81-4620-a140-85938b412a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:11.878989 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-1e489d3c-3b81-4620-a140-85938b412a01 None None] MFA Rules not processed for user `b2ba168683a4433f8a763cb4c250c9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:11.880522 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-1e489d3c-3b81-4620-a140-85938b412a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:12.028199 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:12.070160 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1186/2347] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:12.082586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dbe014-348b-4a21-82a6-62978e6b2234 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:12.088364 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-90dbe014-348b-4a21-82a6-62978e6b2234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:12.134641 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1162/2348] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:12.140084 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e489d3c-3b81-4620-a140-85938b412a01 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:12.149685 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-1e489d3c-3b81-4620-a140-85938b412a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:12.361238 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e489d3c-3b81-4620-a140-85938b412a01 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:12.374037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1160e-ac8c-4095-9265-e666a87a7257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:12.374959 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1160e-ac8c-4095-9265-e666a87a7257 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:12.375359 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1160e-ac8c-4095-9265-e666a87a7257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:12.376792 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1160e-ac8c-4095-9265-e666a87a7257 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:12.377574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1160e-ac8c-4095-9265-e666a87a7257 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:12.379563 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1160e-ac8c-4095-9265-e666a87a7257 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:12.450135 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1163/2349] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:07:12 2022] POST /identity/v3/projects => generated 371 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:12.459052 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1160e-ac8c-4095-9265-e666a87a7257 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:12.472708 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dbe014-348b-4a21-82a6-62978e6b2234 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:12.570760 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc52eab-7c54-4849-a61a-df1f61217c4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:12.571563 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc52eab-7c54-4849-a61a-df1f61217c4c service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:12.572059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc52eab-7c54-4849-a61a-df1f61217c4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:12.572344 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc52eab-7c54-4849-a61a-df1f61217c4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:12.629640 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1160e-ac8c-4095-9265-e666a87a7257 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:12.637619 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0510ba-e59b-435b-8603-ae453d5e1735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:12.639484 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0510ba-e59b-435b-8603-ae453d5e1735 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:12.640196 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0510ba-e59b-435b-8603-ae453d5e1735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:12.640809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0510ba-e59b-435b-8603-ae453d5e1735 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:12.642380 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0510ba-e59b-435b-8603-ae453d5e1735 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:12.647474 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0510ba-e59b-435b-8603-ae453d5e1735 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:12.672079 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc52eab-7c54-4849-a61a-df1f61217c4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:12.673644 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc52eab-7c54-4849-a61a-df1f61217c4c service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:12.679079 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1164/2350] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:07:12 2022] POST /identity/v3/users => generated 292 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:12.686996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0510ba-e59b-435b-8603-ae453d5e1735 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:12.689604 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1187/2351] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:07:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:12.705197 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0510ba-e59b-435b-8603-ae453d5e1735 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:12.710765 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440ed80-8684-4930-90c2-23475073f478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:12.711385 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440ed80-8684-4930-90c2-23475073f478 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:12.711781 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440ed80-8684-4930-90c2-23475073f478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:12.712241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440ed80-8684-4930-90c2-23475073f478 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:12.713380 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4440ed80-8684-4930-90c2-23475073f478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:12.715251 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4440ed80-8684-4930-90c2-23475073f478 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:12.744540 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc52eab-7c54-4849-a61a-df1f61217c4c service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:12.751723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1165/2352] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:12.760106 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440ed80-8684-4930-90c2-23475073f478 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:12.760398 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc52eab-7c54-4849-a61a-df1f61217c4c service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:12.771695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44decbe-4c21-4c42-82c0-5620456848c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:12.772632 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44decbe-4c21-4c42-82c0-5620456848c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:12.772920 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44decbe-4c21-4c42-82c0-5620456848c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:12.773203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44decbe-4c21-4c42-82c0-5620456848c1 admin admin] PATH_INFO: `/v3/users/5158149d96fe42c795a630b301dbd53f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:12.774705 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44decbe-4c21-4c42-82c0-5620456848c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5158149d96fe42c795a630b301dbd53f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:12.775763 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440ed80-8684-4930-90c2-23475073f478 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:12.777407 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44decbe-4c21-4c42-82c0-5620456848c1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:12.787559 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fec5b-a035-40c2-bab6-ac13390d4f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:12.788726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fec5b-a035-40c2-bab6-ac13390d4f9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:12.789118 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fec5b-a035-40c2-bab6-ac13390d4f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:12.789503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fec5b-a035-40c2-bab6-ac13390d4f9a admin admin] PATH_INFO: `/v3/projects/8351057258f14080aa9614468eee2daf/users/b6ea4a1f33eb490bb5aa9aeea78e3c94/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:12.806312 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:12.821374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4fec5b-a035-40c2-bab6-ac13390d4f9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8351057258f14080aa9614468eee2daf, user_id=b6ea4a1f33eb490bb5aa9aeea78e3c94, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:12.829175 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4fec5b-a035-40c2-bab6-ac13390d4f9a admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:12.847638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1166/2353] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:07:12 2022] PUT /identity/v3/projects/8351057258f14080aa9614468eee2daf/users/b6ea4a1f33eb490bb5aa9aeea78e3c94/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:12.852341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fec5b-a035-40c2-bab6-ac13390d4f9a admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:12.879592 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a44decbe-4c21-4c42-82c0-5620456848c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5158149d96fe42c795a630b301dbd53f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:12.889201 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:12.891589 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a44decbe-4c21-4c42-82c0-5620456848c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5158149d96fe42c795a630b301dbd53f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:12.913771 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a44decbe-4c21-4c42-82c0-5620456848c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5158149d96fe42c795a630b301dbd53f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:12.914792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a44decbe-4c21-4c42-82c0-5620456848c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5158149d96fe42c795a630b301dbd53f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:12.915633 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a44decbe-4c21-4c42-82c0-5620456848c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5158149d96fe42c795a630b301dbd53f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:12.938727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1188/2354] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:07:12 2022] DELETE /identity/v3/users/5158149d96fe42c795a630b301dbd53f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:12.943389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d7a03-2943-4be2-94ba-0528930acec7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:12.943761 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d7a03-2943-4be2-94ba-0528930acec7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:12.944052 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d7a03-2943-4be2-94ba-0528930acec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:12.961146 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fec5b-a035-40c2-bab6-ac13390d4f9a admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:12.965178 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-074d7a03-2943-4be2-94ba-0528930acec7 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:12.966499 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-074d7a03-2943-4be2-94ba-0528930acec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:13.046288 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495913e-c365-4935-87ef-534efe5259c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:13.047198 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495913e-c365-4935-87ef-534efe5259c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:13.047494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495913e-c365-4935-87ef-534efe5259c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:13.047938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495913e-c365-4935-87ef-534efe5259c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:13.048709 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9495913e-c365-4935-87ef-534efe5259c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:13.050393 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9495913e-c365-4935-87ef-534efe5259c1 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:13.058233 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1189/2355] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:07:13.062720 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1167/2356] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:13.067577 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d7a03-2943-4be2-94ba-0528930acec7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:13.069094 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-074d7a03-2943-4be2-94ba-0528930acec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:13.071098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495913e-c365-4935-87ef-534efe5259c1 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:13.072996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-9495913e-c365-4935-87ef-534efe5259c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:13.081833 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d7a03-2943-4be2-94ba-0528930acec7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:13.091109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef73882-9314-4856-a4da-9273ce1df976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:13.093228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef73882-9314-4856-a4da-9273ce1df976 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:13.093539 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef73882-9314-4856-a4da-9273ce1df976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:13.093875 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef73882-9314-4856-a4da-9273ce1df976 admin admin] PATH_INFO: `/v3/projects/8351057258f14080aa9614468eee2daf/users/b6ea4a1f33eb490bb5aa9aeea78e3c94/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:13.099484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef73882-9314-4856-a4da-9273ce1df976 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8351057258f14080aa9614468eee2daf, user_id=b6ea4a1f33eb490bb5aa9aeea78e3c94, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:13.106417 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef73882-9314-4856-a4da-9273ce1df976 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:13.120980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1190/2357] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:07:13 2022] PUT /identity/v3/projects/8351057258f14080aa9614468eee2daf/users/b6ea4a1f33eb490bb5aa9aeea78e3c94/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:13.175405 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495913e-c365-4935-87ef-534efe5259c1 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:13.305163 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc01d04-9167-4e85-b762-0469ea940924 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:13.305850 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc01d04-9167-4e85-b762-0469ea940924 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:13.306760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc01d04-9167-4e85-b762-0469ea940924 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:13.307060 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc01d04-9167-4e85-b762-0469ea940924 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:13.402859 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:13.452349 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc01d04-9167-4e85-b762-0469ea940924 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:13.453827 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc01d04-9167-4e85-b762-0469ea940924 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:13.489246 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1168/2358] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:07:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:07:13.965235 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef73882-9314-4856-a4da-9273ce1df976 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:13.967812 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9ef73882-9314-4856-a4da-9273ce1df976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:13.993868 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef73882-9314-4856-a4da-9273ce1df976 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:14.000784 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86eff59-1f81-48c3-b146-06579b34bacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:14.001868 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86eff59-1f81-48c3-b146-06579b34bacb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:14.002246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86eff59-1f81-48c3-b146-06579b34bacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:14.002510 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86eff59-1f81-48c3-b146-06579b34bacb admin admin] PATH_INFO: `/v3/projects/4bb46b1882464c04a476935ff315eb97` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:14.004091 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86eff59-1f81-48c3-b146-06579b34bacb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bb46b1882464c04a476935ff315eb97)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:14.006115 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86eff59-1f81-48c3-b146-06579b34bacb admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:14.094920 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e86eff59-1f81-48c3-b146-06579b34bacb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bb46b1882464c04a476935ff315eb97', 'request_id': 'req-e86eff59-1f81-48c3-b146-06579b34bacb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:14.106081 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e86eff59-1f81-48c3-b146-06579b34bacb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bb46b1882464c04a476935ff315eb97', 'request_id': 'req-e86eff59-1f81-48c3-b146-06579b34bacb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:14.119912 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e86eff59-1f81-48c3-b146-06579b34bacb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bb46b1882464c04a476935ff315eb97', 'request_id': 'req-e86eff59-1f81-48c3-b146-06579b34bacb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:14.156752 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:DELETE=17,keystone:SELECT=80,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:14.171859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e86eff59-1f81-48c3-b146-06579b34bacb admin admin] The token cache is being invalidate because project 4bb46b1882464c04a476935ff315eb97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:07:14.172421 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e86eff59-1f81-48c3-b146-06579b34bacb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:14.173147 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e86eff59-1f81-48c3-b146-06579b34bacb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:14.175488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1191/2359] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:07:13 2022] DELETE /identity/v3/projects/4bb46b1882464c04a476935ff315eb97 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:07:14.182327 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e5b00-72da-46fa-bcab-69cca2882837 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:14.183820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e5b00-72da-46fa-bcab-69cca2882837 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:14.184449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e5b00-72da-46fa-bcab-69cca2882837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:14.208602 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-4b5e5b00-72da-46fa-bcab-69cca2882837 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:14.209849 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-4b5e5b00-72da-46fa-bcab-69cca2882837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:14.292425 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1169/2360] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:14.294807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8059b2-3170-4431-913c-53a2dc9378d8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:14.295574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8059b2-3170-4431-913c-53a2dc9378d8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:14.296184 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8059b2-3170-4431-913c-53a2dc9378d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:14.319101 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-9c8059b2-3170-4431-913c-53a2dc9378d8 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:14.320628 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9c8059b2-3170-4431-913c-53a2dc9378d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:14.335008 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1192/2361] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:14.336303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e5b00-72da-46fa-bcab-69cca2882837 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:14.351779 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e5b00-72da-46fa-bcab-69cca2882837 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:14.361918 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ed0b6-d65b-47be-b685-2225a12caf12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:14.363431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ed0b6-d65b-47be-b685-2225a12caf12 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:14.364164 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ed0b6-d65b-47be-b685-2225a12caf12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:14.364364 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ed0b6-d65b-47be-b685-2225a12caf12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:14.365907 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ed0b6-d65b-47be-b685-2225a12caf12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:07:14.366812 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ed0b6-d65b-47be-b685-2225a12caf12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:14.368372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ed0b6-d65b-47be-b685-2225a12caf12 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:14.388116 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1170/2362] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:07:14.393590 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8059b2-3170-4431-913c-53a2dc9378d8 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:14.403854 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8059b2-3170-4431-913c-53a2dc9378d8 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:14.409906 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee02d7-8d7c-4894-a82b-4f3e7a564357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:14.410774 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee02d7-8d7c-4894-a82b-4f3e7a564357 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:14.410936 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee02d7-8d7c-4894-a82b-4f3e7a564357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:14.411201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee02d7-8d7c-4894-a82b-4f3e7a564357 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:14.412779 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee02d7-8d7c-4894-a82b-4f3e7a564357 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:14.414863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee02d7-8d7c-4894-a82b-4f3e7a564357 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:14.448050 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1193/2363] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:07:14 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:07:14.453131 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ed0b6-d65b-47be-b685-2225a12caf12 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:14.464929 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ed0b6-d65b-47be-b685-2225a12caf12 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:14.535841 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7317c6f-90b2-4b1a-a3c1-507201282e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:14.537117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7317c6f-90b2-4b1a-a3c1-507201282e3e admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:14.537694 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7317c6f-90b2-4b1a-a3c1-507201282e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:14.538330 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7317c6f-90b2-4b1a-a3c1-507201282e3e admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:14.539336 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7317c6f-90b2-4b1a-a3c1-507201282e3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:14.542157 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7317c6f-90b2-4b1a-a3c1-507201282e3e admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:14.572970 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1171/2364] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:07:14 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:14.579748 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee02d7-8d7c-4894-a82b-4f3e7a564357 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:14.597113 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee02d7-8d7c-4894-a82b-4f3e7a564357 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:14.603597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e41cea-bb1e-4bb0-b63c-66f2a1b2ab45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:14.604300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e41cea-bb1e-4bb0-b63c-66f2a1b2ab45 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:14.604591 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e41cea-bb1e-4bb0-b63c-66f2a1b2ab45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:14.604868 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e41cea-bb1e-4bb0-b63c-66f2a1b2ab45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:14.605605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e41cea-bb1e-4bb0-b63c-66f2a1b2ab45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:14.607333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e41cea-bb1e-4bb0-b63c-66f2a1b2ab45 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:14.632643 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1194/2365] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:14.634202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7317c6f-90b2-4b1a-a3c1-507201282e3e admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:14.647772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7317c6f-90b2-4b1a-a3c1-507201282e3e admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:14.652199 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4481ec02-2e52-40b5-8ff3-dcf6c8b97375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:14.653100 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481ec02-2e52-40b5-8ff3-dcf6c8b97375 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:14.653802 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481ec02-2e52-40b5-8ff3-dcf6c8b97375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:14.655034 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481ec02-2e52-40b5-8ff3-dcf6c8b97375 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:14.655907 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4481ec02-2e52-40b5-8ff3-dcf6c8b97375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:14.657598 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4481ec02-2e52-40b5-8ff3-dcf6c8b97375 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:14.674250 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1172/2366] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:14.675834 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e41cea-bb1e-4bb0-b63c-66f2a1b2ab45 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:14.687851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e41cea-bb1e-4bb0-b63c-66f2a1b2ab45 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:14.694999 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ace295-aab6-482a-be7e-000c8663c8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:14.696167 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ace295-aab6-482a-be7e-000c8663c8fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:14.696768 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ace295-aab6-482a-be7e-000c8663c8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:14.697212 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ace295-aab6-482a-be7e-000c8663c8fa admin admin] PATH_INFO: `/v3/projects/98761773ea78490baebe0b3d9414ac45/users/c02d672b8b864940a57d9954fa44c6b2/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:14.718673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ace295-aab6-482a-be7e-000c8663c8fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=98761773ea78490baebe0b3d9414ac45, user_id=c02d672b8b864940a57d9954fa44c6b2, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:14.722075 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ace295-aab6-482a-be7e-000c8663c8fa admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:14.774050 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1195/2367] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:07:14 2022] PUT /identity/v3/projects/98761773ea78490baebe0b3d9414ac45/users/c02d672b8b864940a57d9954fa44c6b2/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:14.776008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cb3c5-1985-470f-bd8d-0ccd538e3338 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:14.776300 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cb3c5-1985-470f-bd8d-0ccd538e3338 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:14.776590 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cb3c5-1985-470f-bd8d-0ccd538e3338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:14.833439 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-5a9cb3c5-1985-470f-bd8d-0ccd538e3338 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:14.841300 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-5a9cb3c5-1985-470f-bd8d-0ccd538e3338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:14.924415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:14.979530 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1173/2368] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:15.008247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ace295-aab6-482a-be7e-000c8663c8fa admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:15.010745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-38ace295-aab6-482a-be7e-000c8663c8fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:15.086926 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ace295-aab6-482a-be7e-000c8663c8fa admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:15.091508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa20e4d-54e3-400c-96b0-a2ced9c274ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:15.092506 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa20e4d-54e3-400c-96b0-a2ced9c274ad service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:15.093106 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa20e4d-54e3-400c-96b0-a2ced9c274ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:15.093631 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa20e4d-54e3-400c-96b0-a2ced9c274ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:15.104842 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa20e4d-54e3-400c-96b0-a2ced9c274ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:15.106825 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa20e4d-54e3-400c-96b0-a2ced9c274ad service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:15.119714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1196/2369] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:07:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:07:23.011749 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7665d1-7f8a-4e11-9397-1448da9f2738 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:23.011749 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7665d1-7f8a-4e11-9397-1448da9f2738 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:23.012365 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7665d1-7f8a-4e11-9397-1448da9f2738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:23.120686 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-ce7665d1-7f8a-4e11-9397-1448da9f2738 None None] MFA Rules not processed for user `b6ea4a1f33eb490bb5aa9aeea78e3c94`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:23.124112 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-ce7665d1-7f8a-4e11-9397-1448da9f2738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:23.358141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1174/2370] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:23 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:23.375017 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa20e4d-54e3-400c-96b0-a2ced9c274ad service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:23.378181 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5aa20e4d-54e3-400c-96b0-a2ced9c274ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:23.546619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa20e4d-54e3-400c-96b0-a2ced9c274ad service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:23.555087 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5783344-7670-4769-95ed-023b513eecf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:23.555773 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5783344-7670-4769-95ed-023b513eecf4 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:23.556105 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5783344-7670-4769-95ed-023b513eecf4 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:23.556383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5783344-7670-4769-95ed-023b513eecf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:23.558619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f5783344-7670-4769-95ed-023b513eecf4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:23.611860 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:23.673889 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5783344-7670-4769-95ed-023b513eecf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:23.677275 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5783344-7670-4769-95ed-023b513eecf4 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:23.703231 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1197/2371] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:07:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:24.457010 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7665d1-7f8a-4e11-9397-1448da9f2738 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:24.462709 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-ce7665d1-7f8a-4e11-9397-1448da9f2738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:24.490905 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7665d1-7f8a-4e11-9397-1448da9f2738 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:24.509061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c7c65-bc5a-4d9d-b2fd-d836e757094b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:24.510370 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c7c65-bc5a-4d9d-b2fd-d836e757094b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:24.511773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c7c65-bc5a-4d9d-b2fd-d836e757094b admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:24.512456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c7c65-bc5a-4d9d-b2fd-d836e757094b admin admin] PATH_INFO: `/v3/users/eb9fa9855586408f9c50b3d7999965dc` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:24.516904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c7c65-bc5a-4d9d-b2fd-d836e757094b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb9fa9855586408f9c50b3d7999965dc)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:24.519834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c7c65-bc5a-4d9d-b2fd-d836e757094b admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:24.684388 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-a10c7c65-bc5a-4d9d-b2fd-d836e757094b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb9fa9855586408f9c50b3d7999965dc'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:24.705960 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-a10c7c65-bc5a-4d9d-b2fd-d836e757094b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb9fa9855586408f9c50b3d7999965dc'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:24.706878 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-a10c7c65-bc5a-4d9d-b2fd-d836e757094b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb9fa9855586408f9c50b3d7999965dc'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:24.708248 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-a10c7c65-bc5a-4d9d-b2fd-d836e757094b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb9fa9855586408f9c50b3d7999965dc'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:24.764369 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-a10c7c65-bc5a-4d9d-b2fd-d836e757094b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb9fa9855586408f9c50b3d7999965dc'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:24.846316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1175/2372] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:07:24 2022] DELETE /identity/v3/users/eb9fa9855586408f9c50b3d7999965dc => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:24.852474 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e0895-c1c2-41b7-87e9-8651942413a4 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:24.853035 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e0895-c1c2-41b7-87e9-8651942413a4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:24.853473 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e0895-c1c2-41b7-87e9-8651942413a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:24.937968 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8e5e0895-c1c2-41b7-87e9-8651942413a4 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:24.945134 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8e5e0895-c1c2-41b7-87e9-8651942413a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:25.176741 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1198/2373] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:25.242235 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c7c65-bc5a-4d9d-b2fd-d836e757094b admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:25.246368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-a10c7c65-bc5a-4d9d-b2fd-d836e757094b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:25.360313 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:25.434946 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c7c65-bc5a-4d9d-b2fd-d836e757094b admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:25.442408 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a028c1-9fa2-4cc9-861e-7b6bf58287f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:25.443091 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a028c1-9fa2-4cc9-861e-7b6bf58287f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:25.443381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a028c1-9fa2-4cc9-861e-7b6bf58287f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:25.443968 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a028c1-9fa2-4cc9-861e-7b6bf58287f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:25.507661 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a028c1-9fa2-4cc9-861e-7b6bf58287f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:25.511604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a028c1-9fa2-4cc9-861e-7b6bf58287f9 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:25.542819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1176/2374] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:07:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:25.855492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e0895-c1c2-41b7-87e9-8651942413a4 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:25.857255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8e5e0895-c1c2-41b7-87e9-8651942413a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:25.875597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e0895-c1c2-41b7-87e9-8651942413a4 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:25.889605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68e3f6-43ac-4176-8f14-f25a879cc171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:25.891090 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68e3f6-43ac-4176-8f14-f25a879cc171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:25.891675 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68e3f6-43ac-4176-8f14-f25a879cc171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:25.892141 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68e3f6-43ac-4176-8f14-f25a879cc171 admin admin] PATH_INFO: `/v3/users/eadf0a646bb048818edef5d3c51c7b7f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:25.894219 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68e3f6-43ac-4176-8f14-f25a879cc171 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eadf0a646bb048818edef5d3c51c7b7f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:25.897096 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68e3f6-43ac-4176-8f14-f25a879cc171 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:25.986320 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ca68e3f6-43ac-4176-8f14-f25a879cc171 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eadf0a646bb048818edef5d3c51c7b7f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:25.998987 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ca68e3f6-43ac-4176-8f14-f25a879cc171 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eadf0a646bb048818edef5d3c51c7b7f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:26.041271 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ca68e3f6-43ac-4176-8f14-f25a879cc171 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eadf0a646bb048818edef5d3c51c7b7f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:26.042860 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ca68e3f6-43ac-4176-8f14-f25a879cc171 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eadf0a646bb048818edef5d3c51c7b7f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:26.043579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ca68e3f6-43ac-4176-8f14-f25a879cc171 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eadf0a646bb048818edef5d3c51c7b7f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:26.087264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1199/2375] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:07:25 2022] DELETE /identity/v3/users/eadf0a646bb048818edef5d3c51c7b7f => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:26.093561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8009a-ea9f-475f-8e66-697a9e8de852 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:26.095198 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8009a-ea9f-475f-8e66-697a9e8de852 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:26.096109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8009a-ea9f-475f-8e66-697a9e8de852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:26.186004 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-e3e8009a-ea9f-475f-8e66-697a9e8de852 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:26.190371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-e3e8009a-ea9f-475f-8e66-697a9e8de852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:26.405710 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f651c-905f-4992-bed0-5cbbde770003 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:26.406033 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f651c-905f-4992-bed0-5cbbde770003 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:26.406373 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f651c-905f-4992-bed0-5cbbde770003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:26.432606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1177/2376] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:26.459142 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8009a-ea9f-475f-8e66-697a9e8de852 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:26.463596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-e3e8009a-ea9f-475f-8e66-697a9e8de852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:26.468132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-850f651c-905f-4992-bed0-5cbbde770003 None None] MFA Rules not processed for user `b2ba168683a4433f8a763cb4c250c9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:26.482221 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-850f651c-905f-4992-bed0-5cbbde770003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:26.705264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:26.710093 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8009a-ea9f-475f-8e66-697a9e8de852 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:26.722161 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76bf84-a4d2-445e-a2c9-c6aadf2499b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:26.723792 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76bf84-a4d2-445e-a2c9-c6aadf2499b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:26.726303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76bf84-a4d2-445e-a2c9-c6aadf2499b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:26.726977 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76bf84-a4d2-445e-a2c9-c6aadf2499b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:26.769985 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1200/2377] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:26.772871 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76bf84-a4d2-445e-a2c9-c6aadf2499b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:26.774743 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76bf84-a4d2-445e-a2c9-c6aadf2499b7 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:26.807215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1178/2378] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:07:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:27.958353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f651c-905f-4992-bed0-5cbbde770003 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:27.962016 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-850f651c-905f-4992-bed0-5cbbde770003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:28.015483 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f651c-905f-4992-bed0-5cbbde770003 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:28.027205 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47dec68-d802-4184-bff9-f3f6a0f08ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:28.028101 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47dec68-d802-4184-bff9-f3f6a0f08ca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:28.032209 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47dec68-d802-4184-bff9-f3f6a0f08ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:28.035547 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47dec68-d802-4184-bff9-f3f6a0f08ca8 admin admin] PATH_INFO: `/v3/projects/68590393cacc4bf1bfe65c4343d5a1da` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:28.037078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47dec68-d802-4184-bff9-f3f6a0f08ca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68590393cacc4bf1bfe65c4343d5a1da)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:28.055009 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47dec68-d802-4184-bff9-f3f6a0f08ca8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:28.123837 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e47dec68-d802-4184-bff9-f3f6a0f08ca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68590393cacc4bf1bfe65c4343d5a1da', 'request_id': 'req-e47dec68-d802-4184-bff9-f3f6a0f08ca8'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:28.146683 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e47dec68-d802-4184-bff9-f3f6a0f08ca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68590393cacc4bf1bfe65c4343d5a1da', 'request_id': 'req-e47dec68-d802-4184-bff9-f3f6a0f08ca8'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:28.153419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e47dec68-d802-4184-bff9-f3f6a0f08ca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68590393cacc4bf1bfe65c4343d5a1da', 'request_id': 'req-e47dec68-d802-4184-bff9-f3f6a0f08ca8'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:28.170967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e47dec68-d802-4184-bff9-f3f6a0f08ca8 admin admin] The token cache is being invalidate because project 68590393cacc4bf1bfe65c4343d5a1da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:07:28.171197 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e47dec68-d802-4184-bff9-f3f6a0f08ca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:28.171942 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e47dec68-d802-4184-bff9-f3f6a0f08ca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:28.173620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1201/2379] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:07:27 2022] DELETE /identity/v3/projects/68590393cacc4bf1bfe65c4343d5a1da => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:29.000192 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a0e23-da2d-4138-adb8-0b7e77f9efb6 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:29.001107 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a0e23-da2d-4138-adb8-0b7e77f9efb6 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:29.001545 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a0e23-da2d-4138-adb8-0b7e77f9efb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:29.079464 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-922a0e23-da2d-4138-adb8-0b7e77f9efb6 None None] MFA Rules not processed for user `c02d672b8b864940a57d9954fa44c6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:29.082256 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-922a0e23-da2d-4138-adb8-0b7e77f9efb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:29.202343 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47dec68-d802-4184-bff9-f3f6a0f08ca8 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:29.205798 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e47dec68-d802-4184-bff9-f3f6a0f08ca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:29.392125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1179/2380] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:29.395120 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a0e23-da2d-4138-adb8-0b7e77f9efb6 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:29.404355 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-922a0e23-da2d-4138-adb8-0b7e77f9efb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:29.451303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:29.500966 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47dec68-d802-4184-bff9-f3f6a0f08ca8 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:29.506923 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07589996-9f24-44b6-89f0-1ee290606804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:29.508983 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07589996-9f24-44b6-89f0-1ee290606804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:29.510048 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07589996-9f24-44b6-89f0-1ee290606804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:29.510456 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07589996-9f24-44b6-89f0-1ee290606804 admin admin] PATH_INFO: `/v3/projects/8a31bff554fd4c6f8f84fa87f14ec8c8` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:29.511945 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07589996-9f24-44b6-89f0-1ee290606804 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a31bff554fd4c6f8f84fa87f14ec8c8)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:29.520115 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07589996-9f24-44b6-89f0-1ee290606804 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:29.688592 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a0e23-da2d-4138-adb8-0b7e77f9efb6 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:29.712209 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-07589996-9f24-44b6-89f0-1ee290606804 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a31bff554fd4c6f8f84fa87f14ec8c8', 'request_id': 'req-07589996-9f24-44b6-89f0-1ee290606804'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:29.731262 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-07589996-9f24-44b6-89f0-1ee290606804 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a31bff554fd4c6f8f84fa87f14ec8c8', 'request_id': 'req-07589996-9f24-44b6-89f0-1ee290606804'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:29.763445 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-07589996-9f24-44b6-89f0-1ee290606804 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a31bff554fd4c6f8f84fa87f14ec8c8', 'request_id': 'req-07589996-9f24-44b6-89f0-1ee290606804'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:29.800022 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-07589996-9f24-44b6-89f0-1ee290606804 admin admin] The token cache is being invalidate because project 8a31bff554fd4c6f8f84fa87f14ec8c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:07:29.800372 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-07589996-9f24-44b6-89f0-1ee290606804 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:29.803479 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-07589996-9f24-44b6-89f0-1ee290606804 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:07:29.806241 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1202/2381] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:07:29 2022] DELETE /identity/v3/projects/8a31bff554fd4c6f8f84fa87f14ec8c8 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:29.835086 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76531dd-3f38-4b8a-a55b-598c819fe436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:29.836139 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76531dd-3f38-4b8a-a55b-598c819fe436 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:29.836534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76531dd-3f38-4b8a-a55b-598c819fe436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:29.836777 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76531dd-3f38-4b8a-a55b-598c819fe436 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:29.837553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76531dd-3f38-4b8a-a55b-598c819fe436 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:29.839911 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76531dd-3f38-4b8a-a55b-598c819fe436 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:29.926179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1180/2382] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:07:29 2022] POST /identity/v3/projects => generated 383 bytes in 532 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:07:29.931417 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07589996-9f24-44b6-89f0-1ee290606804 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:29.933588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-07589996-9f24-44b6-89f0-1ee290606804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:29.957906 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:29.965126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07589996-9f24-44b6-89f0-1ee290606804 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:30.126887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd83d25-1b8b-47fb-83f9-b8e2bd051853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:30.127768 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd83d25-1b8b-47fb-83f9-b8e2bd051853 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:30.128194 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd83d25-1b8b-47fb-83f9-b8e2bd051853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:30.128566 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd83d25-1b8b-47fb-83f9-b8e2bd051853 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:30.129500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd83d25-1b8b-47fb-83f9-b8e2bd051853 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:30.131655 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd83d25-1b8b-47fb-83f9-b8e2bd051853 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:30.187441 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1203/2383] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:07:29 2022] POST /identity/v3/users => generated 298 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:07:30.192528 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76531dd-3f38-4b8a-a55b-598c819fe436 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:30.222975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76531dd-3f38-4b8a-a55b-598c819fe436 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:30.245896 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1927d2-db81-4ae5-8143-808cd429af36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:30.248133 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1927d2-db81-4ae5-8143-808cd429af36 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:30.248750 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1927d2-db81-4ae5-8143-808cd429af36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:30.249297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1927d2-db81-4ae5-8143-808cd429af36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:30.250713 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1927d2-db81-4ae5-8143-808cd429af36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:30.253742 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1927d2-db81-4ae5-8143-808cd429af36 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:30.291747 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1181/2384] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:30.292222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd83d25-1b8b-47fb-83f9-b8e2bd051853 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:30.324193 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd83d25-1b8b-47fb-83f9-b8e2bd051853 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:30.332206 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a74d1d-6232-40fd-8c94-86d825d26ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:30.333845 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a74d1d-6232-40fd-8c94-86d825d26ecd admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:30.334492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a74d1d-6232-40fd-8c94-86d825d26ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:30.335061 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a74d1d-6232-40fd-8c94-86d825d26ecd admin admin] PATH_INFO: `/v3/projects/13d695a79cfa4d82ad819d0de3fdd7c6/users/6be3a0f98dd44b84b04f390a677da96f/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:30.365504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a74d1d-6232-40fd-8c94-86d825d26ecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=13d695a79cfa4d82ad819d0de3fdd7c6, user_id=6be3a0f98dd44b84b04f390a677da96f, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:30.370349 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a74d1d-6232-40fd-8c94-86d825d26ecd admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:30.402355 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1204/2385] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:07:30 2022] PUT /identity/v3/projects/13d695a79cfa4d82ad819d0de3fdd7c6/users/6be3a0f98dd44b84b04f390a677da96f/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:30.407511 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1927d2-db81-4ae5-8143-808cd429af36 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:30.430108 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1927d2-db81-4ae5-8143-808cd429af36 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:30.562828 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d6f24-26bd-4ac3-bffa-86eb71d80b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:30.564241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d6f24-26bd-4ac3-bffa-86eb71d80b73 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:30.567985 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d6f24-26bd-4ac3-bffa-86eb71d80b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:30.568566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d6f24-26bd-4ac3-bffa-86eb71d80b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:30.571044 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d6f24-26bd-4ac3-bffa-86eb71d80b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:30.578167 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d6f24-26bd-4ac3-bffa-86eb71d80b73 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:30.617148 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a74d1d-6232-40fd-8c94-86d825d26ecd admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:30.618660 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1182/2386] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:07:30.636021 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a74d1d-6232-40fd-8c94-86d825d26ecd admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:30.648368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa952f5d-696d-45c4-932a-2ae7bf3327e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:30.649355 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa952f5d-696d-45c4-932a-2ae7bf3327e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:30.649794 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa952f5d-696d-45c4-932a-2ae7bf3327e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:30.651547 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa952f5d-696d-45c4-932a-2ae7bf3327e2 admin admin] PATH_INFO: `/v3/projects/13d695a79cfa4d82ad819d0de3fdd7c6/users/6be3a0f98dd44b84b04f390a677da96f/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:30.657177 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa952f5d-696d-45c4-932a-2ae7bf3327e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13d695a79cfa4d82ad819d0de3fdd7c6, user_id=6be3a0f98dd44b84b04f390a677da96f, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:30.662588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa952f5d-696d-45c4-932a-2ae7bf3327e2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:30.680771 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1205/2387] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:07:30 2022] PUT /identity/v3/projects/13d695a79cfa4d82ad819d0de3fdd7c6/users/6be3a0f98dd44b84b04f390a677da96f/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:07:31.981573 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ce28f-0b7b-4cac-8851-cc82a9098658 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:31.982538 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ce28f-0b7b-4cac-8851-cc82a9098658 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:31.982989 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ce28f-0b7b-4cac-8851-cc82a9098658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:32.003448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-f83ce28f-0b7b-4cac-8851-cc82a9098658 None None] MFA Rules not processed for user `42c96a85400941818b658cca8cb56fec`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:32.004684 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f83ce28f-0b7b-4cac-8851-cc82a9098658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:32.038083 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:32.089079 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1183/2388] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:38.688745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d35c9d-40a8-4873-989e-48ce26d2697c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:38.689674 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d35c9d-40a8-4873-989e-48ce26d2697c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:38.689674 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d35c9d-40a8-4873-989e-48ce26d2697c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:38.745458 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e7d35c9d-40a8-4873-989e-48ce26d2697c None None] MFA Rules not processed for user `61f6dbd4328e459eb7d4f029551648bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:38.746840 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e7d35c9d-40a8-4873-989e-48ce26d2697c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:38.874219 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1206/2389] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:07:40.986286 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40e619-9964-4ed4-aab4-9f4b99a7f6fa None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:40.987010 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40e619-9964-4ed4-aab4-9f4b99a7f6fa None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:40.987010 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40e619-9964-4ed4-aab4-9f4b99a7f6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:41.034326 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-4a40e619-9964-4ed4-aab4-9f4b99a7f6fa None None] MFA Rules not processed for user `6be3a0f98dd44b84b04f390a677da96f`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:41.035937 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-4a40e619-9964-4ed4-aab4-9f4b99a7f6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:41.185382 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1184/2390] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:41.212333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d35c9d-40a8-4873-989e-48ce26d2697c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:41.217313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e7d35c9d-40a8-4873-989e-48ce26d2697c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:41.422988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:41.449217 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d35c9d-40a8-4873-989e-48ce26d2697c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:41.458460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52957b01-63c9-4b3e-9dfd-9c365e04d2fd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f4c9fc949c4a2c951b755234848cc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:41.459930 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52957b01-63c9-4b3e-9dfd-9c365e04d2fd service glance] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:41.460651 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52957b01-63c9-4b3e-9dfd-9c365e04d2fd service glance] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:41.462345 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52957b01-63c9-4b3e-9dfd-9c365e04d2fd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:41.480760 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-52957b01-63c9-4b3e-9dfd-9c365e04d2fd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:41.618993 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52957b01-63c9-4b3e-9dfd-9c365e04d2fd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:41.620592 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52957b01-63c9-4b3e-9dfd-9c365e04d2fd service glance] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:41.643985 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1207/2391] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:07:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:07:41.982753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f588690-7353-46f8-8a52-05842868cac2 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:41.983451 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f588690-7353-46f8-8a52-05842868cac2 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:41.983748 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f588690-7353-46f8-8a52-05842868cac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:42.082653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-3f588690-7353-46f8-8a52-05842868cac2 None None] MFA Rules not processed for user `3e06d1b0e1a9472db764f4be7e42fdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:42.084440 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-3f588690-7353-46f8-8a52-05842868cac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:42.248675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1185/2392] 10.222.0.76 () {58 vars in 1009 bytes} [Thu May 26 18:07:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:07:42.249382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52957b01-63c9-4b3e-9dfd-9c365e04d2fd service glance] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:42.251827 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-52957b01-63c9-4b3e-9dfd-9c365e04d2fd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:42.603834 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52957b01-63c9-4b3e-9dfd-9c365e04d2fd service glance] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:42.631745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa43f0-c656-43c5-8eeb-41e738c1b7d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ca774e864ca44cfb187e38bc8a8f6ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:42.633719 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa43f0-c656-43c5-8eeb-41e738c1b7d4 service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:42.634448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa43f0-c656-43c5-8eeb-41e738c1b7d4 service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:42.635223 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa43f0-c656-43c5-8eeb-41e738c1b7d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:42.674925 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa43f0-c656-43c5-8eeb-41e738c1b7d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:42.678294 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa43f0-c656-43c5-8eeb-41e738c1b7d4 service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:42.744010 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1208/2393] 10.222.0.76 () {60 vars in 1384 bytes} [Thu May 26 18:07:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:07:49.812083 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f588690-7353-46f8-8a52-05842868cac2 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:07:49.813607 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-3f588690-7353-46f8-8a52-05842868cac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:49.864981 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:49.992546 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f588690-7353-46f8-8a52-05842868cac2 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:07:49.999841 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb54f9-43e2-47d7-917f-2348f6aeb1c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '596f126382e3440fb3cf7c357eef889f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:07:50.000647 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb54f9-43e2-47d7-917f-2348f6aeb1c6 service placement] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:50.001058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb54f9-43e2-47d7-917f-2348f6aeb1c6 service placement] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:50.001419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb54f9-43e2-47d7-917f-2348f6aeb1c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:50.003977 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-2acb54f9-43e2-47d7-917f-2348f6aeb1c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:50.025642 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb54f9-43e2-47d7-917f-2348f6aeb1c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:07:50.030521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb54f9-43e2-47d7-917f-2348f6aeb1c6 service placement] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:07:50.043303 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1186/2394] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:07:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:07:52.699644 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=70 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:07:56.321385 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85716b4d-d423-4616-ab5b-d4b4936e079c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:07:56.322153 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85716b4d-d423-4616-ab5b-d4b4936e079c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:07:56.322409 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85716b4d-d423-4616-ab5b-d4b4936e079c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:07:56.342487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-85716b4d-d423-4616-ab5b-d4b4936e079c None None] MFA Rules not processed for user `c02d672b8b864940a57d9954fa44c6b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:07:56.343791 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-85716b4d-d423-4616-ab5b-d4b4936e079c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:07:56.353416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1209/2395] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:07:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:08:00.039420 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=28 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:08:00.839217 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a57f16-9f99-4c36-a974-fa24ebf1cf0d None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:00.839628 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a57f16-9f99-4c36-a974-fa24ebf1cf0d None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:00.839999 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a57f16-9f99-4c36-a974-fa24ebf1cf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:00.902847 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-20a57f16-9f99-4c36-a974-fa24ebf1cf0d None None] MFA Rules not processed for user `3e06d1b0e1a9472db764f4be7e42fdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:08:00.905518 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-20a57f16-9f99-4c36-a974-fa24ebf1cf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:08:00.921154 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1187/2396] 10.222.0.76 () {58 vars in 1009 bytes} [Thu May 26 18:08:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:08:00.936044 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85716b4d-d423-4616-ab5b-d4b4936e079c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:08:00.957967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85716b4d-d423-4616-ab5b-d4b4936e079c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:08:00.974349 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a695f-ca42-472e-ab2b-2c7c13811aef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ca774e864ca44cfb187e38bc8a8f6ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:08:00.975388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a695f-ca42-472e-ab2b-2c7c13811aef service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:00.975913 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a695f-ca42-472e-ab2b-2c7c13811aef service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:00.976599 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a695f-ca42-472e-ab2b-2c7c13811aef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:01.020004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a695f-ca42-472e-ab2b-2c7c13811aef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:08:01.039413 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a695f-ca42-472e-ab2b-2c7c13811aef service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:08:01.081233 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1210/2397] 10.222.0.76 () {60 vars in 1384 bytes} [Thu May 26 18:08:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 18:08:10.878689 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:08:11.058032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=9 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:08:15.561147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a57f16-9f99-4c36-a974-fa24ebf1cf0d None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:08:15.562760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-20a57f16-9f99-4c36-a974-fa24ebf1cf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:08:15.620706 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a57f16-9f99-4c36-a974-fa24ebf1cf0d None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:08:15.624737 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453b041-0044-4efa-b0b4-b5515c58cf58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:08:15.625387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453b041-0044-4efa-b0b4-b5515c58cf58 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:15.625671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453b041-0044-4efa-b0b4-b5515c58cf58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:15.625946 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453b041-0044-4efa-b0b4-b5515c58cf58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:15.628140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-d453b041-0044-4efa-b0b4-b5515c58cf58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:08:15.695332 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d453b041-0044-4efa-b0b4-b5515c58cf58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:08:15.697193 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d453b041-0044-4efa-b0b4-b5515c58cf58 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:08:15.708655 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1188/2398] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:08:15 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:08:25.704211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=50 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:08:27.639998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a695f-ca42-472e-ab2b-2c7c13811aef service swift] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:08:27.642177 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-582a695f-ca42-472e-ab2b-2c7c13811aef service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:08:27.657497 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a695f-ca42-472e-ab2b-2c7c13811aef service swift] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:08:27.662751 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b0cd0-9dba-41d3-bbad-c78ed1f73ebb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:08:27.663719 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b0cd0-9dba-41d3-bbad-c78ed1f73ebb service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:27.664029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b0cd0-9dba-41d3-bbad-c78ed1f73ebb service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:27.664509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b0cd0-9dba-41d3-bbad-c78ed1f73ebb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:27.676796 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-933b0cd0-9dba-41d3-bbad-c78ed1f73ebb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:08:27.767934 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b0cd0-9dba-41d3-bbad-c78ed1f73ebb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:08:27.769528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b0cd0-9dba-41d3-bbad-c78ed1f73ebb service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:08:27.782144 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1211/2399] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:08:27 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:08:34.835742 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e40e9-16a0-43c7-806e-a1804392a0e4 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:34.836335 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e40e9-16a0-43c7-806e-a1804392a0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:34.836335 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e40e9-16a0-43c7-806e-a1804392a0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:34.853746 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-b12e40e9-16a0-43c7-806e-a1804392a0e4 None None] MFA Rules not processed for user `dfbe23f3a0294148a88f4758d8470665`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:08:34.855007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-b12e40e9-16a0-43c7-806e-a1804392a0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:08:34.913659 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1189/2400] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:08:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:08:37.490662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b0cd0-9dba-41d3-bbad-c78ed1f73ebb service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:08:37.500413 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b0cd0-9dba-41d3-bbad-c78ed1f73ebb service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:08:37.504419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d31f0d-6a5a-4f71-80b1-2f750cea50d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:08:37.505127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d31f0d-6a5a-4f71-80b1-2f750cea50d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:08:37.505400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d31f0d-6a5a-4f71-80b1-2f750cea50d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:08:37.505684 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d31f0d-6a5a-4f71-80b1-2f750cea50d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:08:37.516279 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d31f0d-6a5a-4f71-80b1-2f750cea50d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:08:37.517724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d31f0d-6a5a-4f71-80b1-2f750cea50d3 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:08:37.528163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1212/2401] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:08:37 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:08:44.906935 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=25 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:08:47.523345 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=36 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:23.952927 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602c9b1-e912-4b38-89d8-863f0e8e4fc4 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:23.953510 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602c9b1-e912-4b38-89d8-863f0e8e4fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:23.953510 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602c9b1-e912-4b38-89d8-863f0e8e4fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:23.971883 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-9602c9b1-e912-4b38-89d8-863f0e8e4fc4 None None] MFA Rules not processed for user `61f6dbd4328e459eb7d4f029551648bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:23.973109 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-9602c9b1-e912-4b38-89d8-863f0e8e4fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:23.980284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1190/2402] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:09:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:33.966491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:46.210381 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d31f0d-6a5a-4f71-80b1-2f750cea50d3 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:46.211809 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c1d31f0d-6a5a-4f71-80b1-2f750cea50d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:46.274913 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d31f0d-6a5a-4f71-80b1-2f750cea50d3 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:46.279438 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ead64-a5ed-4b1c-affb-50d593ad6e25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:46.280103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ead64-a5ed-4b1c-affb-50d593ad6e25 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:46.280401 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ead64-a5ed-4b1c-affb-50d593ad6e25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:46.280663 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ead64-a5ed-4b1c-affb-50d593ad6e25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:46.282572 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a33ead64-a5ed-4b1c-affb-50d593ad6e25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:46.360954 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ead64-a5ed-4b1c-affb-50d593ad6e25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:46.362348 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ead64-a5ed-4b1c-affb-50d593ad6e25 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:46.375083 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1213/2403] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:09:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:09:46.545594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602c9b1-e912-4b38-89d8-863f0e8e4fc4 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:46.555200 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602c9b1-e912-4b38-89d8-863f0e8e4fc4 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:46.559365 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b42c97-882d-4572-82e5-ce4934aadd8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:46.560009 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b42c97-882d-4572-82e5-ce4934aadd8a service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:46.560297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b42c97-882d-4572-82e5-ce4934aadd8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:46.560629 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b42c97-882d-4572-82e5-ce4934aadd8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:46.562390 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-d5b42c97-882d-4572-82e5-ce4934aadd8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:46.577585 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b42c97-882d-4572-82e5-ce4934aadd8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:46.579081 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b42c97-882d-4572-82e5-ce4934aadd8a service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:46.591021 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1191/2404] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:09:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:09:56.179781 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f72169-fb55-47b3-92f9-6d7c32032913 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:56.180367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f72169-fb55-47b3-92f9-6d7c32032913 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:56.180367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f72169-fb55-47b3-92f9-6d7c32032913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:56.200620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-96f72169-fb55-47b3-92f9-6d7c32032913 None None] MFA Rules not processed for user `dfbe23f3a0294148a88f4758d8470665`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:09:56.201860 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-96f72169-fb55-47b3-92f9-6d7c32032913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:56.209088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1214/2405] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:09:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:09:56.586387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=6 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:09:56.946419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b42c97-882d-4572-82e5-ce4934aadd8a service cinder] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:56.958491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b42c97-882d-4572-82e5-ce4934aadd8a service cinder] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:56.964653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd5bc1-623d-4f4f-9e4d-54cdd137bb8b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '596f126382e3440fb3cf7c357eef889f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:56.965346 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd5bc1-623d-4f4f-9e4d-54cdd137bb8b service placement] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:56.965633 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd5bc1-623d-4f4f-9e4d-54cdd137bb8b service placement] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:56.965912 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd5bc1-623d-4f4f-9e4d-54cdd137bb8b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:56.969001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-14cd5bc1-623d-4f4f-9e4d-54cdd137bb8b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:56.991518 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd5bc1-623d-4f4f-9e4d-54cdd137bb8b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:56.993502 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd5bc1-623d-4f4f-9e4d-54cdd137bb8b service placement] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:57.006690 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1192/2406] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:09:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:09:59.118162 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f72169-fb55-47b3-92f9-6d7c32032913 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:09:59.130957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f72169-fb55-47b3-92f9-6d7c32032913 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:09:59.134821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624be9ff-d7fa-4b98-9fca-d17ccded364a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:09:59.136205 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624be9ff-d7fa-4b98-9fca-d17ccded364a service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:09:59.136438 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624be9ff-d7fa-4b98-9fca-d17ccded364a service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:09:59.136746 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624be9ff-d7fa-4b98-9fca-d17ccded364a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:09:59.138814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-624be9ff-d7fa-4b98-9fca-d17ccded364a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:09:59.150357 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624be9ff-d7fa-4b98-9fca-d17ccded364a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:09:59.152903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624be9ff-d7fa-4b98-9fca-d17ccded364a service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:09:59.167210 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1215/2407] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:09:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:01.505670 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd5bc1-623d-4f4f-9e4d-54cdd137bb8b service placement] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:01.515233 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd5bc1-623d-4f4f-9e4d-54cdd137bb8b service placement] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:01.518845 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e905b0e-ef24-4c41-a6c9-a08549449277 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:01.519516 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e905b0e-ef24-4c41-a6c9-a08549449277 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:01.519806 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e905b0e-ef24-4c41-a6c9-a08549449277 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:01.520084 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e905b0e-ef24-4c41-a6c9-a08549449277 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:01.521833 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-2e905b0e-ef24-4c41-a6c9-a08549449277 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:01.583788 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e905b0e-ef24-4c41-a6c9-a08549449277 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:01.585436 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e905b0e-ef24-4c41-a6c9-a08549449277 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:01.595625 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1193/2408] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:10:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:04.234716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624be9ff-d7fa-4b98-9fca-d17ccded364a service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:04.236226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-624be9ff-d7fa-4b98-9fca-d17ccded364a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:04.254308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624be9ff-d7fa-4b98-9fca-d17ccded364a service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:04.260187 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cff090-651b-418a-8e84-4ec1c4f335b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:04.260872 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cff090-651b-418a-8e84-4ec1c4f335b3 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:04.261155 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cff090-651b-418a-8e84-4ec1c4f335b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:04.261434 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cff090-651b-418a-8e84-4ec1c4f335b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:04.263421 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-45cff090-651b-418a-8e84-4ec1c4f335b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:04.275590 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cff090-651b-418a-8e84-4ec1c4f335b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:04.277292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cff090-651b-418a-8e84-4ec1c4f335b3 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:04.292137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1216/2409] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:10:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:09.408289 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6e56d-7215-458d-bec9-6f8d331a4903 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:09.408929 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6e56d-7215-458d-bec9-6f8d331a4903 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:09.408929 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6e56d-7215-458d-bec9-6f8d331a4903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:09.432337 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-47d6e56d-7215-458d-bec9-6f8d331a4903 None None] MFA Rules not processed for user `f6b5113ea0f4484ba91a5f0e6184ca35`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:09.433883 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-47d6e56d-7215-458d-bec9-6f8d331a4903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:09.441649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1194/2410] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:10:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:10.110539 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cff090-651b-418a-8e84-4ec1c4f335b3 service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:10.115226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=70 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:10.124999 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cff090-651b-418a-8e84-4ec1c4f335b3 service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:10.128928 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e2980-7a36-483a-8d07-894cb438e7e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:10.129612 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e2980-7a36-483a-8d07-894cb438e7e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:10.129893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e2980-7a36-483a-8d07-894cb438e7e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:10.130426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e2980-7a36-483a-8d07-894cb438e7e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:10.132560 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b87e2980-7a36-483a-8d07-894cb438e7e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:10.144622 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e2980-7a36-483a-8d07-894cb438e7e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:10.146107 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e2980-7a36-483a-8d07-894cb438e7e6 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:10.158532 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1217/2411] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:10:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:11.109425 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6e56d-7215-458d-bec9-6f8d331a4903 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:11.121330 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6e56d-7215-458d-bec9-6f8d331a4903 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:11.125667 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5dc61-506c-495a-ae8c-9e5eb790f7c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:11.126477 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5dc61-506c-495a-ae8c-9e5eb790f7c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:11.126947 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5dc61-506c-495a-ae8c-9e5eb790f7c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:11.127425 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5dc61-506c-495a-ae8c-9e5eb790f7c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:11.134905 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-b4b5dc61-506c-495a-ae8c-9e5eb790f7c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:11.152990 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5dc61-506c-495a-ae8c-9e5eb790f7c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:11.156423 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5dc61-506c-495a-ae8c-9e5eb790f7c9 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:11.174046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1195/2412] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:10:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:14.563884 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e2980-7a36-483a-8d07-894cb438e7e6 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:14.566241 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b87e2980-7a36-483a-8d07-894cb438e7e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:14.599105 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e2980-7a36-483a-8d07-894cb438e7e6 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:14.610989 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a9709-cf34-4d36-a60e-9ab94db26634 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:14.612186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a9709-cf34-4d36-a60e-9ab94db26634 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:14.612700 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a9709-cf34-4d36-a60e-9ab94db26634 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:14.613210 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a9709-cf34-4d36-a60e-9ab94db26634 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:14.616303 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ee8a9709-cf34-4d36-a60e-9ab94db26634 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:14.641156 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a9709-cf34-4d36-a60e-9ab94db26634 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:14.643557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a9709-cf34-4d36-a60e-9ab94db26634 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:14.662681 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1218/2413] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:10:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:21.165534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=43 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:24.654279 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=11 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:25.031694 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027476f0-58c2-493b-8cc6-c34046fa9e71 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:25.032476 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027476f0-58c2-493b-8cc6-c34046fa9e71 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:25.032755 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027476f0-58c2-493b-8cc6-c34046fa9e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:25.057513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-027476f0-58c2-493b-8cc6-c34046fa9e71 None None] MFA Rules not processed for user `3e06d1b0e1a9472db764f4be7e42fdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:25.059077 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-027476f0-58c2-493b-8cc6-c34046fa9e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:25.068135 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1196/2414] 10.222.0.76 () {58 vars in 967 bytes} [Thu May 26 18:10:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:25.127743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a9709-cf34-4d36-a60e-9ab94db26634 service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:25.155430 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a9709-cf34-4d36-a60e-9ab94db26634 service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:25.162400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ca59c-38bb-4139-869c-ad5339219782 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ca774e864ca44cfb187e38bc8a8f6ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:25.169092 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ca59c-38bb-4139-869c-ad5339219782 service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:25.169092 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ca59c-38bb-4139-869c-ad5339219782 service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:25.169092 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ca59c-38bb-4139-869c-ad5339219782 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:25.178924 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ca59c-38bb-4139-869c-ad5339219782 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:25.183276 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ca59c-38bb-4139-869c-ad5339219782 service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:25.198452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1219/2415] 10.222.0.76 () {60 vars in 1384 bytes} [Thu May 26 18:10:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:10:26.038419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027476f0-58c2-493b-8cc6-c34046fa9e71 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:26.064469 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027476f0-58c2-493b-8cc6-c34046fa9e71 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:26.074558 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a5aea-d20f-4856-ab0a-f498b8fabe75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:26.076169 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a5aea-d20f-4856-ab0a-f498b8fabe75 service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:26.076692 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a5aea-d20f-4856-ab0a-f498b8fabe75 service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:26.077213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a5aea-d20f-4856-ab0a-f498b8fabe75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:26.080999 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-188a5aea-d20f-4856-ab0a-f498b8fabe75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:26.106088 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a5aea-d20f-4856-ab0a-f498b8fabe75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:26.109372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a5aea-d20f-4856-ab0a-f498b8fabe75 service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:26.130780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1197/2416] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:10:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:30.527967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ca59c-38bb-4139-869c-ad5339219782 service swift] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:30.546858 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ca59c-38bb-4139-869c-ad5339219782 service swift] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:30.553796 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81b744-172c-4106-bcf1-16a7ca8674f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '596f126382e3440fb3cf7c357eef889f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:30.554578 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81b744-172c-4106-bcf1-16a7ca8674f0 service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:30.554905 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81b744-172c-4106-bcf1-16a7ca8674f0 service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:30.555193 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81b744-172c-4106-bcf1-16a7ca8674f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:30.558501 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5e81b744-172c-4106-bcf1-16a7ca8674f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:30.573917 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e81b744-172c-4106-bcf1-16a7ca8674f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:30.575694 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e81b744-172c-4106-bcf1-16a7ca8674f0 service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:30.592393 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1220/2417] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:10:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:10:35.563735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a5aea-d20f-4856-ab0a-f498b8fabe75 service nova] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:35.567270 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-188a5aea-d20f-4856-ab0a-f498b8fabe75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:35.611892 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a5aea-d20f-4856-ab0a-f498b8fabe75 service nova] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:35.628580 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a4d7b-2275-41c2-888a-d95b536a555f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '596f126382e3440fb3cf7c357eef889f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:35.633081 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a4d7b-2275-41c2-888a-d95b536a555f service placement] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:35.633885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a4d7b-2275-41c2-888a-d95b536a555f service placement] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:35.634396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a4d7b-2275-41c2-888a-d95b536a555f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:35.643110 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-ca9a4d7b-2275-41c2-888a-d95b536a555f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:35.675553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a4d7b-2275-41c2-888a-d95b536a555f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:35.679001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a4d7b-2275-41c2-888a-d95b536a555f service placement] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:35.704218 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81b744-172c-4106-bcf1-16a7ca8674f0 service placement] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:35.704732 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1198/2418] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:10:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:10:35.793221 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81b744-172c-4106-bcf1-16a7ca8674f0 service placement] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:35.797951 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf3856-09ad-4e8a-b70d-eaf5a40f5b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:35.799094 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf3856-09ad-4e8a-b70d-eaf5a40f5b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:35.799376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf3856-09ad-4e8a-b70d-eaf5a40f5b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:35.799649 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf3856-09ad-4e8a-b70d-eaf5a40f5b2e admin admin] PATH_INFO: `/v3/users/c02d672b8b864940a57d9954fa44c6b2` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:35.800937 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf3856-09ad-4e8a-b70d-eaf5a40f5b2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c02d672b8b864940a57d9954fa44c6b2)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:35.802948 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf3856-09ad-4e8a-b70d-eaf5a40f5b2e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:35.872625 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-13bf3856-09ad-4e8a-b70d-eaf5a40f5b2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c02d672b8b864940a57d9954fa44c6b2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:35.882389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-13bf3856-09ad-4e8a-b70d-eaf5a40f5b2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c02d672b8b864940a57d9954fa44c6b2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:35.896680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-13bf3856-09ad-4e8a-b70d-eaf5a40f5b2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c02d672b8b864940a57d9954fa44c6b2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:35.897406 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-13bf3856-09ad-4e8a-b70d-eaf5a40f5b2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c02d672b8b864940a57d9954fa44c6b2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:35.898009 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-13bf3856-09ad-4e8a-b70d-eaf5a40f5b2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c02d672b8b864940a57d9954fa44c6b2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:35.911300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1221/2419] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:10:35 2022] DELETE /identity/v3/users/c02d672b8b864940a57d9954fa44c6b2 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:10:36.106003 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a4d7b-2275-41c2-888a-d95b536a555f service placement] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:36.118510 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a4d7b-2275-41c2-888a-d95b536a555f service placement] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:36.184101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caffd59d-3998-4ed2-a311-8daa0cc3b6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:36.184989 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffd59d-3998-4ed2-a311-8daa0cc3b6f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:36.185385 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffd59d-3998-4ed2-a311-8daa0cc3b6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:36.185764 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffd59d-3998-4ed2-a311-8daa0cc3b6f7 admin admin] PATH_INFO: `/v3/projects/98761773ea78490baebe0b3d9414ac45` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:36.187039 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caffd59d-3998-4ed2-a311-8daa0cc3b6f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98761773ea78490baebe0b3d9414ac45)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:36.188957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caffd59d-3998-4ed2-a311-8daa0cc3b6f7 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:36.239164 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-caffd59d-3998-4ed2-a311-8daa0cc3b6f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98761773ea78490baebe0b3d9414ac45', 'request_id': 'req-caffd59d-3998-4ed2-a311-8daa0cc3b6f7'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:36.248011 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-caffd59d-3998-4ed2-a311-8daa0cc3b6f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98761773ea78490baebe0b3d9414ac45', 'request_id': 'req-caffd59d-3998-4ed2-a311-8daa0cc3b6f7'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:36.254581 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-caffd59d-3998-4ed2-a311-8daa0cc3b6f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98761773ea78490baebe0b3d9414ac45', 'request_id': 'req-caffd59d-3998-4ed2-a311-8daa0cc3b6f7'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:36.266334 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-caffd59d-3998-4ed2-a311-8daa0cc3b6f7 admin admin] The token cache is being invalidate because project 98761773ea78490baebe0b3d9414ac45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:10:36.266623 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-caffd59d-3998-4ed2-a311-8daa0cc3b6f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:36.267340 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-caffd59d-3998-4ed2-a311-8daa0cc3b6f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:36.268844 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1199/2420] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:10:36 2022] DELETE /identity/v3/projects/98761773ea78490baebe0b3d9414ac45 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:10:36.272884 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf3856-09ad-4e8a-b70d-eaf5a40f5b2e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:36.274364 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-13bf3856-09ad-4e8a-b70d-eaf5a40f5b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:36.288709 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf3856-09ad-4e8a-b70d-eaf5a40f5b2e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:36.344659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:36.355077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee112a-35c0-4bfe-bfc5-3feb5ba47a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:36.356525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee112a-35c0-4bfe-bfc5-3feb5ba47a88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:36.356525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee112a-35c0-4bfe-bfc5-3feb5ba47a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:36.356525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee112a-35c0-4bfe-bfc5-3feb5ba47a88 admin admin] PATH_INFO: `/v3/users/6be3a0f98dd44b84b04f390a677da96f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:36.357720 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee112a-35c0-4bfe-bfc5-3feb5ba47a88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6be3a0f98dd44b84b04f390a677da96f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:36.359718 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee112a-35c0-4bfe-bfc5-3feb5ba47a88 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:36.410160 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6dee112a-35c0-4bfe-bfc5-3feb5ba47a88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6be3a0f98dd44b84b04f390a677da96f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:36.418177 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6dee112a-35c0-4bfe-bfc5-3feb5ba47a88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6be3a0f98dd44b84b04f390a677da96f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:36.434150 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6dee112a-35c0-4bfe-bfc5-3feb5ba47a88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6be3a0f98dd44b84b04f390a677da96f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:36.434926 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6dee112a-35c0-4bfe-bfc5-3feb5ba47a88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6be3a0f98dd44b84b04f390a677da96f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:36.435535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6dee112a-35c0-4bfe-bfc5-3feb5ba47a88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6be3a0f98dd44b84b04f390a677da96f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:36.449488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1222/2421] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:10:36 2022] DELETE /identity/v3/users/6be3a0f98dd44b84b04f390a677da96f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:10:36.674868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caffd59d-3998-4ed2-a311-8daa0cc3b6f7 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:36.676435 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-caffd59d-3998-4ed2-a311-8daa0cc3b6f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:36.691618 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caffd59d-3998-4ed2-a311-8daa0cc3b6f7 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:36.765729 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:36.782639 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b0120-0c1b-45b6-bec3-9cde77d4a41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:36.784000 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b0120-0c1b-45b6-bec3-9cde77d4a41e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:36.784756 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b0120-0c1b-45b6-bec3-9cde77d4a41e admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:36.785537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b0120-0c1b-45b6-bec3-9cde77d4a41e admin admin] PATH_INFO: `/v3/projects/13d695a79cfa4d82ad819d0de3fdd7c6` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:36.789820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b0120-0c1b-45b6-bec3-9cde77d4a41e admin admin] RBAC: Authorizing `identity:delete_project(project_id=13d695a79cfa4d82ad819d0de3fdd7c6)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:36.792610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b0120-0c1b-45b6-bec3-9cde77d4a41e admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:36.806008 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee112a-35c0-4bfe-bfc5-3feb5ba47a88 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:36.808473 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6dee112a-35c0-4bfe-bfc5-3feb5ba47a88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:36.867482 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-c50b0120-0c1b-45b6-bec3-9cde77d4a41e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13d695a79cfa4d82ad819d0de3fdd7c6', 'request_id': 'req-c50b0120-0c1b-45b6-bec3-9cde77d4a41e'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:36.876359 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-c50b0120-0c1b-45b6-bec3-9cde77d4a41e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13d695a79cfa4d82ad819d0de3fdd7c6', 'request_id': 'req-c50b0120-0c1b-45b6-bec3-9cde77d4a41e'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:36.877441 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee112a-35c0-4bfe-bfc5-3feb5ba47a88 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:36.885382 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-c50b0120-0c1b-45b6-bec3-9cde77d4a41e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13d695a79cfa4d82ad819d0de3fdd7c6', 'request_id': 'req-c50b0120-0c1b-45b6-bec3-9cde77d4a41e'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:36.898506 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-c50b0120-0c1b-45b6-bec3-9cde77d4a41e admin admin] The token cache is being invalidate because project 13d695a79cfa4d82ad819d0de3fdd7c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:10:36.898801 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-c50b0120-0c1b-45b6-bec3-9cde77d4a41e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:36.899487 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-c50b0120-0c1b-45b6-bec3-9cde77d4a41e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:36.906603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1200/2422] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:10:36 2022] DELETE /identity/v3/projects/13d695a79cfa4d82ad819d0de3fdd7c6 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:10:36.956655 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27278568-292f-4bf2-a9a4-957235950ecf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:36.956940 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27278568-292f-4bf2-a9a4-957235950ecf service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:36.957247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27278568-292f-4bf2-a9a4-957235950ecf service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:36.957442 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27278568-292f-4bf2-a9a4-957235950ecf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:36.959740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-27278568-292f-4bf2-a9a4-957235950ecf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:37.039137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:37.074687 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27278568-292f-4bf2-a9a4-957235950ecf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:37.076366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27278568-292f-4bf2-a9a4-957235950ecf service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:37.100812 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1223/2423] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:10:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:10:41.917379 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840dfe1-04a3-4ae9-bce1-ee727061509c None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:41.918065 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840dfe1-04a3-4ae9-bce1-ee727061509c None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:41.918065 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840dfe1-04a3-4ae9-bce1-ee727061509c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:41.944462 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-f840dfe1-04a3-4ae9-bce1-ee727061509c None None] MFA Rules not processed for user `42c96a85400941818b658cca8cb56fec`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:41.947446 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f840dfe1-04a3-4ae9-bce1-ee727061509c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:42.066336 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1201/2424] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:10:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:42.897691 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27278568-292f-4bf2-a9a4-957235950ecf service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:42.900080 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-27278568-292f-4bf2-a9a4-957235950ecf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:42.916353 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27278568-292f-4bf2-a9a4-957235950ecf service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:42.921532 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e2220-72e4-4c7a-99d5-d52e0e391450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:42.922834 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e2220-72e4-4c7a-99d5-d52e0e391450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:42.923261 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e2220-72e4-4c7a-99d5-d52e0e391450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:42.923648 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e2220-72e4-4c7a-99d5-d52e0e391450 admin admin] PATH_INFO: `/v3/users/61f6dbd4328e459eb7d4f029551648bb` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:42.925279 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e2220-72e4-4c7a-99d5-d52e0e391450 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61f6dbd4328e459eb7d4f029551648bb)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:42.927563 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e2220-72e4-4c7a-99d5-d52e0e391450 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:42.994279 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b91e2220-72e4-4c7a-99d5-d52e0e391450 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61f6dbd4328e459eb7d4f029551648bb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:43.006107 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b91e2220-72e4-4c7a-99d5-d52e0e391450 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61f6dbd4328e459eb7d4f029551648bb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:43.026976 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b91e2220-72e4-4c7a-99d5-d52e0e391450 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61f6dbd4328e459eb7d4f029551648bb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:43.027824 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b91e2220-72e4-4c7a-99d5-d52e0e391450 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61f6dbd4328e459eb7d4f029551648bb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:43.028678 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b91e2220-72e4-4c7a-99d5-d52e0e391450 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61f6dbd4328e459eb7d4f029551648bb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:43.045811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1224/2425] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:10:42 2022] DELETE /identity/v3/users/61f6dbd4328e459eb7d4f029551648bb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:10:43.319521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f840dfe1-04a3-4ae9-bce1-ee727061509c None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:43.342640 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f840dfe1-04a3-4ae9-bce1-ee727061509c None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:43.465739 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bfec0-4a8c-4b86-b513-31efac7b02e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:43.468375 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bfec0-4a8c-4b86-b513-31efac7b02e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:43.468882 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bfec0-4a8c-4b86-b513-31efac7b02e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:43.469407 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bfec0-4a8c-4b86-b513-31efac7b02e1 admin admin] PATH_INFO: `/v3/projects/8c49a9a21f1346a0bc8d774afc7951a2` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:43.471741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bfec0-4a8c-4b86-b513-31efac7b02e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c49a9a21f1346a0bc8d774afc7951a2)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:43.475690 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bfec0-4a8c-4b86-b513-31efac7b02e1 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:43.542538 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:43.564042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-620bfec0-4a8c-4b86-b513-31efac7b02e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c49a9a21f1346a0bc8d774afc7951a2', 'request_id': 'req-620bfec0-4a8c-4b86-b513-31efac7b02e1'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:43.577450 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-620bfec0-4a8c-4b86-b513-31efac7b02e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c49a9a21f1346a0bc8d774afc7951a2', 'request_id': 'req-620bfec0-4a8c-4b86-b513-31efac7b02e1'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:43.586321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-620bfec0-4a8c-4b86-b513-31efac7b02e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c49a9a21f1346a0bc8d774afc7951a2', 'request_id': 'req-620bfec0-4a8c-4b86-b513-31efac7b02e1'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:43.601748 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-620bfec0-4a8c-4b86-b513-31efac7b02e1 admin admin] The token cache is being invalidate because project 8c49a9a21f1346a0bc8d774afc7951a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:10:43.602053 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-620bfec0-4a8c-4b86-b513-31efac7b02e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:43.602769 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-620bfec0-4a8c-4b86-b513-31efac7b02e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:10:43.605106 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1202/2426] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:10:43 2022] DELETE /identity/v3/projects/8c49a9a21f1346a0bc8d774afc7951a2 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:10:43.611357 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb3395-0544-455e-89e9-090c62053da4 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:43.611907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb3395-0544-455e-89e9-090c62053da4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:43.612302 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb3395-0544-455e-89e9-090c62053da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:43.636706 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-88eb3395-0544-455e-89e9-090c62053da4 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:43.637978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-88eb3395-0544-455e-89e9-090c62053da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:43.727449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1225/2427] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:10:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:43.739565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81351be7-b97c-4d21-9f93-446f0531983f None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:43.739843 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81351be7-b97c-4d21-9f93-446f0531983f None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:43.740143 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81351be7-b97c-4d21-9f93-446f0531983f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:43.763315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-81351be7-b97c-4d21-9f93-446f0531983f None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:43.764598 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-81351be7-b97c-4d21-9f93-446f0531983f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:43.775870 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1203/2428] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:10:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:43.782355 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb3395-0544-455e-89e9-090c62053da4 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:43.795064 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb3395-0544-455e-89e9-090c62053da4 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:43.799901 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd24b4-df78-4228-b766-ab52c04c3e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:43.800494 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd24b4-df78-4228-b766-ab52c04c3e35 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:43.800778 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd24b4-df78-4228-b766-ab52c04c3e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:43.801053 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd24b4-df78-4228-b766-ab52c04c3e35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:43.801797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd24b4-df78-4228-b766-ab52c04c3e35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:10:43.802326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd24b4-df78-4228-b766-ab52c04c3e35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:43.804052 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd24b4-df78-4228-b766-ab52c04c3e35 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:43.819984 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1226/2429] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:10:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:10:43.826945 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81351be7-b97c-4d21-9f93-446f0531983f None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:43.839031 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81351be7-b97c-4d21-9f93-446f0531983f None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:43.846068 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096727d-2a08-42f2-b84f-c718b0287e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:43.847075 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096727d-2a08-42f2-b84f-c718b0287e4f admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:43.847431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096727d-2a08-42f2-b84f-c718b0287e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:43.847765 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096727d-2a08-42f2-b84f-c718b0287e4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:43.848746 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096727d-2a08-42f2-b84f-c718b0287e4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:43.851133 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096727d-2a08-42f2-b84f-c718b0287e4f admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:43.893334 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1204/2430] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:10:43 2022] POST /identity/v3/projects => generated 387 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:10:43.897838 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd24b4-df78-4228-b766-ab52c04c3e35 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:43.909846 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd24b4-df78-4228-b766-ab52c04c3e35 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:43.974548 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:44.009734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f237f1-97d9-4ea9-af43-1c6e2d942a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:44.010480 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f237f1-97d9-4ea9-af43-1c6e2d942a15 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:44.010770 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f237f1-97d9-4ea9-af43-1c6e2d942a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:44.011051 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f237f1-97d9-4ea9-af43-1c6e2d942a15 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:44.011764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f237f1-97d9-4ea9-af43-1c6e2d942a15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:44.013771 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f237f1-97d9-4ea9-af43-1c6e2d942a15 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:44.036269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1227/2431] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:10:43 2022] POST /identity/v3/users => generated 294 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:10:44.045220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096727d-2a08-42f2-b84f-c718b0287e4f admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:44.060188 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096727d-2a08-42f2-b84f-c718b0287e4f admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:44.064880 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f797f-2934-4b0c-9edd-f09c60340ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:44.065578 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f797f-2934-4b0c-9edd-f09c60340ffe admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:44.065866 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f797f-2934-4b0c-9edd-f09c60340ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:44.066141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f797f-2934-4b0c-9edd-f09c60340ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:44.066918 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f797f-2934-4b0c-9edd-f09c60340ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:44.068551 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f797f-2934-4b0c-9edd-f09c60340ffe admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:44.080361 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1205/2432] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:10:44.084603 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f237f1-97d9-4ea9-af43-1c6e2d942a15 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:44.095667 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f237f1-97d9-4ea9-af43-1c6e2d942a15 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:44.100467 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc623c8c-a153-4e28-a4d4-48b59d194b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:44.101157 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc623c8c-a153-4e28-a4d4-48b59d194b0d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:44.101437 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc623c8c-a153-4e28-a4d4-48b59d194b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:44.101707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc623c8c-a153-4e28-a4d4-48b59d194b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:44.102481 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc623c8c-a153-4e28-a4d4-48b59d194b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:44.104813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc623c8c-a153-4e28-a4d4-48b59d194b0d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:44.116106 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1228/2433] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:10:44.121602 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f797f-2934-4b0c-9edd-f09c60340ffe admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:44.134768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f797f-2934-4b0c-9edd-f09c60340ffe admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:44.141495 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96deb51b-7b33-4727-8926-4cd0f96999c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:44.142784 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96deb51b-7b33-4727-8926-4cd0f96999c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:44.143268 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96deb51b-7b33-4727-8926-4cd0f96999c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:44.143777 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96deb51b-7b33-4727-8926-4cd0f96999c9 admin admin] PATH_INFO: `/v3/projects/1b13e82b890d4bedbd8ecb6bcbd603c6/users/1b262e67242c4384ba5bf0356a08679e/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:44.164712 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96deb51b-7b33-4727-8926-4cd0f96999c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b13e82b890d4bedbd8ecb6bcbd603c6, user_id=1b262e67242c4384ba5bf0356a08679e, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:44.168167 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96deb51b-7b33-4727-8926-4cd0f96999c9 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:44.184306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1206/2434] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:10:44 2022] PUT /identity/v3/projects/1b13e82b890d4bedbd8ecb6bcbd603c6/users/1b262e67242c4384ba5bf0356a08679e/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:10:44.190149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df432517-d377-4b90-ab40-387cf5adbaea None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:44.190544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df432517-d377-4b90-ab40-387cf5adbaea None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:44.190917 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df432517-d377-4b90-ab40-387cf5adbaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:44.211243 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-df432517-d377-4b90-ab40-387cf5adbaea None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:44.213442 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-df432517-d377-4b90-ab40-387cf5adbaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:44.283521 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1229/2435] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:10:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:44.294285 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96deb51b-7b33-4727-8926-4cd0f96999c9 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:44.296222 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-96deb51b-7b33-4727-8926-4cd0f96999c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:44.366814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96deb51b-7b33-4727-8926-4cd0f96999c9 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:44.371789 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b132c0-f03d-4290-92f3-bb758174c5a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:44.372462 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b132c0-f03d-4290-92f3-bb758174c5a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:44.372742 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b132c0-f03d-4290-92f3-bb758174c5a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:44.373019 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b132c0-f03d-4290-92f3-bb758174c5a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:44.384990 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b132c0-f03d-4290-92f3-bb758174c5a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:44.386748 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b132c0-f03d-4290-92f3-bb758174c5a3 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:44.398701 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1207/2436] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:10:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:51.747253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791f24f-df1b-4e00-8f5a-43d49148bd25 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:51.747762 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791f24f-df1b-4e00-8f5a-43d49148bd25 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:51.747866 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791f24f-df1b-4e00-8f5a-43d49148bd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:51.787085 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a791f24f-df1b-4e00-8f5a-43d49148bd25 None None] MFA Rules not processed for user `1b262e67242c4384ba5bf0356a08679e`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:51.788385 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a791f24f-df1b-4e00-8f5a-43d49148bd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:51.873937 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1230/2437] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:10:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:10:51.888951 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b132c0-f03d-4290-92f3-bb758174c5a3 service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:10:51.891305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f4b132c0-f03d-4290-92f3-bb758174c5a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:51.970605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:10:51.994136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b132c0-f03d-4290-92f3-bb758174c5a3 service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:10:52.001790 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb6108-c146-46a5-8da0-8d08c8a7e49e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:10:52.002505 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb6108-c146-46a5-8da0-8d08c8a7e49e service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:52.002794 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb6108-c146-46a5-8da0-8d08c8a7e49e service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:52.003099 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb6108-c146-46a5-8da0-8d08c8a7e49e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:52.013312 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb6108-c146-46a5-8da0-8d08c8a7e49e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:10:52.014963 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb6108-c146-46a5-8da0-8d08c8a7e49e service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:10:52.026073 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1208/2438] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:10:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:10:53.597866 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77db695-6998-4e54-9b84-c5366756ac8c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:10:53.598473 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77db695-6998-4e54-9b84-c5366756ac8c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:10:53.598800 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77db695-6998-4e54-9b84-c5366756ac8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:10:53.621695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b77db695-6998-4e54-9b84-c5366756ac8c None None] MFA Rules not processed for user `1b262e67242c4384ba5bf0356a08679e`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:10:53.623422 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b77db695-6998-4e54-9b84-c5366756ac8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:10:53.640757 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1231/2439] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:10:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:02.020161 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=6 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:03.614823 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:04.545188 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ea0ab-0836-46ae-8504-58a6e9b291ee None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:04.545917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ea0ab-0836-46ae-8504-58a6e9b291ee None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:04.545917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ea0ab-0836-46ae-8504-58a6e9b291ee None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:04.548216 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1209/2440] 10.222.0.76 () {54 vars in 921 bytes} [Thu May 26 18:11:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:11:04.556630 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64557172-8a97-4867-b6b1-aea090fe5eaa None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:04.556920 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64557172-8a97-4867-b6b1-aea090fe5eaa None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:04.557368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64557172-8a97-4867-b6b1-aea090fe5eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:04.583762 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-64557172-8a97-4867-b6b1-aea090fe5eaa None None] MFA Rules not processed for user `8df66990a3734209a350c8b6768d259f`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:04.586054 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-64557172-8a97-4867-b6b1-aea090fe5eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:04.598391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1232/2441] 10.222.0.76 () {58 vars in 1018 bytes} [Thu May 26 18:11:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:04.611667 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ea0ab-0836-46ae-8504-58a6e9b291ee None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:04.625262 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ea0ab-0836-46ae-8504-58a6e9b291ee None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:04.629599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a38af-6f51-4a4e-9871-004b8b3b9892 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:04.630729 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a38af-6f51-4a4e-9871-004b8b3b9892 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:04.632265 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a38af-6f51-4a4e-9871-004b8b3b9892 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:04.632775 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a38af-6f51-4a4e-9871-004b8b3b9892 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:04.643146 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a38af-6f51-4a4e-9871-004b8b3b9892 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:04.645487 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a38af-6f51-4a4e-9871-004b8b3b9892 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:04.659377 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1210/2442] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:11:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:04.751107 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28d599-6be3-41c1-91ba-70d86be3c147 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:04.751770 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28d599-6be3-41c1-91ba-70d86be3c147 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:04.751770 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28d599-6be3-41c1-91ba-70d86be3c147 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:04.753282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1233/2443] 10.222.0.76 () {54 vars in 921 bytes} [Thu May 26 18:11:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:11:04.760301 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347028a-619a-4065-aaa4-91ee6cec8584 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:04.760671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347028a-619a-4065-aaa4-91ee6cec8584 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:04.761024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347028a-619a-4065-aaa4-91ee6cec8584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:04.801119 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-8347028a-619a-4065-aaa4-91ee6cec8584 None None] MFA Rules not processed for user `1899c317083d40f5b490cd9eb63fe521`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:04.802381 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-8347028a-619a-4065-aaa4-91ee6cec8584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:04.813259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1211/2444] 10.222.0.76 () {58 vars in 1018 bytes} [Thu May 26 18:11:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:04.826669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28d599-6be3-41c1-91ba-70d86be3c147 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:04.839460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28d599-6be3-41c1-91ba-70d86be3c147 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:04.843215 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f15fb-04c9-47f6-bf18-953eebd49c91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:04.843906 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f15fb-04c9-47f6-bf18-953eebd49c91 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:04.844188 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f15fb-04c9-47f6-bf18-953eebd49c91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:04.844463 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f15fb-04c9-47f6-bf18-953eebd49c91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:04.856965 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f15fb-04c9-47f6-bf18-953eebd49c91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:04.858481 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f15fb-04c9-47f6-bf18-953eebd49c91 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:04.870020 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1234/2445] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:11:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:08.584681 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347028a-619a-4065-aaa4-91ee6cec8584 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:08.598544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347028a-619a-4065-aaa4-91ee6cec8584 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:08.604668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1208803-4638-4817-b500-0ca5720dd4aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:08.605802 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1208803-4638-4817-b500-0ca5720dd4aa service nova] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:08.606163 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1208803-4638-4817-b500-0ca5720dd4aa service nova] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:08.606554 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1208803-4638-4817-b500-0ca5720dd4aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:08.608703 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-e1208803-4638-4817-b500-0ca5720dd4aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:08.627481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1208803-4638-4817-b500-0ca5720dd4aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:08.629279 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1208803-4638-4817-b500-0ca5720dd4aa service nova] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:08.662427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1212/2446] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:11:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:11:13.600449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f15fb-04c9-47f6-bf18-953eebd49c91 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:13.601864 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f30f15fb-04c9-47f6-bf18-953eebd49c91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:13.615457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f15fb-04c9-47f6-bf18-953eebd49c91 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:13.619998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3f37e-bf32-4cb6-ab4a-43a517b58697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:13.620665 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3f37e-bf32-4cb6-ab4a-43a517b58697 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:13.620922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3f37e-bf32-4cb6-ab4a-43a517b58697 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:13.621200 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3f37e-bf32-4cb6-ab4a-43a517b58697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:13.622984 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d3a3f37e-bf32-4cb6-ab4a-43a517b58697 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:13.635551 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a3f37e-bf32-4cb6-ab4a-43a517b58697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:13.636985 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a3f37e-bf32-4cb6-ab4a-43a517b58697 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:13.647362 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1235/2447] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:11:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:17.287872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1208803-4638-4817-b500-0ca5720dd4aa service nova] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:17.291275 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-e1208803-4638-4817-b500-0ca5720dd4aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:17.319863 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1208803-4638-4817-b500-0ca5720dd4aa service nova] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:17.325658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6203e1-2557-4e4a-924c-6f37aac6f3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:17.326617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6203e1-2557-4e4a-924c-6f37aac6f3bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:17.326959 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6203e1-2557-4e4a-924c-6f37aac6f3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:17.327172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6203e1-2557-4e4a-924c-6f37aac6f3bb admin admin] PATH_INFO: `/v3/users/dfbe23f3a0294148a88f4758d8470665` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:17.328376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6203e1-2557-4e4a-924c-6f37aac6f3bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfbe23f3a0294148a88f4758d8470665)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:17.330358 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6203e1-2557-4e4a-924c-6f37aac6f3bb admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:17.388433 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-5e6203e1-2557-4e4a-924c-6f37aac6f3bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfbe23f3a0294148a88f4758d8470665'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:17.398234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-5e6203e1-2557-4e4a-924c-6f37aac6f3bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfbe23f3a0294148a88f4758d8470665'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:17.398901 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-5e6203e1-2557-4e4a-924c-6f37aac6f3bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfbe23f3a0294148a88f4758d8470665'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:17.399815 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-5e6203e1-2557-4e4a-924c-6f37aac6f3bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfbe23f3a0294148a88f4758d8470665'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:17.409129 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-5e6203e1-2557-4e4a-924c-6f37aac6f3bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfbe23f3a0294148a88f4758d8470665'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:17.432383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1213/2448] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:11:17 2022] DELETE /identity/v3/users/dfbe23f3a0294148a88f4758d8470665 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:17.632097 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3f37e-bf32-4cb6-ab4a-43a517b58697 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:17.646110 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3f37e-bf32-4cb6-ab4a-43a517b58697 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:17.711282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6d4d1-0cec-4061-b973-98cbf5e984de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:17.712771 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6d4d1-0cec-4061-b973-98cbf5e984de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:17.712771 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6d4d1-0cec-4061-b973-98cbf5e984de admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:17.713198 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6d4d1-0cec-4061-b973-98cbf5e984de admin admin] PATH_INFO: `/v3/projects/eac4a6151fbb44d1b0b61636a97f56b6` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:17.714308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6d4d1-0cec-4061-b973-98cbf5e984de admin admin] RBAC: Authorizing `identity:delete_project(project_id=eac4a6151fbb44d1b0b61636a97f56b6)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:17.717499 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6d4d1-0cec-4061-b973-98cbf5e984de admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:17.779333 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bbe6d4d1-0cec-4061-b973-98cbf5e984de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eac4a6151fbb44d1b0b61636a97f56b6', 'request_id': 'req-bbe6d4d1-0cec-4061-b973-98cbf5e984de'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:17.792163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bbe6d4d1-0cec-4061-b973-98cbf5e984de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eac4a6151fbb44d1b0b61636a97f56b6', 'request_id': 'req-bbe6d4d1-0cec-4061-b973-98cbf5e984de'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:17.800271 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bbe6d4d1-0cec-4061-b973-98cbf5e984de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eac4a6151fbb44d1b0b61636a97f56b6', 'request_id': 'req-bbe6d4d1-0cec-4061-b973-98cbf5e984de'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:17.818690 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bbe6d4d1-0cec-4061-b973-98cbf5e984de admin admin] The token cache is being invalidate because project eac4a6151fbb44d1b0b61636a97f56b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:11:17.819569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bbe6d4d1-0cec-4061-b973-98cbf5e984de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:17.821262 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bbe6d4d1-0cec-4061-b973-98cbf5e984de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:17.822959 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1236/2449] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:11:17 2022] DELETE /identity/v3/projects/eac4a6151fbb44d1b0b61636a97f56b6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:17.828287 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e9e23-9b01-41f0-a468-3a2fe9ede5b3 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:17.828593 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e9e23-9b01-41f0-a468-3a2fe9ede5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:17.828840 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e9e23-9b01-41f0-a468-3a2fe9ede5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:17.853492 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-613e9e23-9b01-41f0-a468-3a2fe9ede5b3 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:17.854836 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-613e9e23-9b01-41f0-a468-3a2fe9ede5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:17.933955 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1214/2450] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:17.944381 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c26782-0979-460e-8e29-92cd44a25ae3 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:17.944936 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c26782-0979-460e-8e29-92cd44a25ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:17.945441 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c26782-0979-460e-8e29-92cd44a25ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:17.974829 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-05c26782-0979-460e-8e29-92cd44a25ae3 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:17.976685 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-05c26782-0979-460e-8e29-92cd44a25ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:17.986449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1237/2451] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:17.991240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e9e23-9b01-41f0-a468-3a2fe9ede5b3 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:18.004768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e9e23-9b01-41f0-a468-3a2fe9ede5b3 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:18.013814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3db5c-b094-4c95-9738-b3b5aa15bebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:18.015107 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3db5c-b094-4c95-9738-b3b5aa15bebc admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:18.015628 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3db5c-b094-4c95-9738-b3b5aa15bebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:18.016184 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3db5c-b094-4c95-9738-b3b5aa15bebc admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:18.017632 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3db5c-b094-4c95-9738-b3b5aa15bebc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:11:18.018173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3db5c-b094-4c95-9738-b3b5aa15bebc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:18.020019 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3db5c-b094-4c95-9738-b3b5aa15bebc admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:18.037074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1215/2452] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:11:18.042530 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c26782-0979-460e-8e29-92cd44a25ae3 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:18.052523 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c26782-0979-460e-8e29-92cd44a25ae3 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:18.058062 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598846db-ca87-46a4-8b87-a94cedc63e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:18.059074 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598846db-ca87-46a4-8b87-a94cedc63e8d admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:18.059603 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598846db-ca87-46a4-8b87-a94cedc63e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:18.059928 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598846db-ca87-46a4-8b87-a94cedc63e8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:18.060826 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598846db-ca87-46a4-8b87-a94cedc63e8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:18.063494 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598846db-ca87-46a4-8b87-a94cedc63e8d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:18.104649 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1238/2453] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:11:18 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:11:18.109185 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3db5c-b094-4c95-9738-b3b5aa15bebc admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:18.120465 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3db5c-b094-4c95-9738-b3b5aa15bebc admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:18.130850 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:18.186683 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f717c72-372f-4679-a704-97415b19ff72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:18.187361 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f717c72-372f-4679-a704-97415b19ff72 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:18.187646 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f717c72-372f-4679-a704-97415b19ff72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:18.187923 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f717c72-372f-4679-a704-97415b19ff72 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:18.188619 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f717c72-372f-4679-a704-97415b19ff72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:18.190591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f717c72-372f-4679-a704-97415b19ff72 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:18.220363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1216/2454] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:11:18 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:11:18.224183 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598846db-ca87-46a4-8b87-a94cedc63e8d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:18.235448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598846db-ca87-46a4-8b87-a94cedc63e8d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:18.240321 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b33411-8c0a-4e6f-814b-32a5d9461cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:18.241026 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b33411-8c0a-4e6f-814b-32a5d9461cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:18.241313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b33411-8c0a-4e6f-814b-32a5d9461cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:18.241591 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b33411-8c0a-4e6f-814b-32a5d9461cc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:18.242346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b33411-8c0a-4e6f-814b-32a5d9461cc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:18.244196 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b33411-8c0a-4e6f-814b-32a5d9461cc1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:18.254979 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1239/2455] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:11:18.259730 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f717c72-372f-4679-a704-97415b19ff72 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:18.269645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f717c72-372f-4679-a704-97415b19ff72 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:18.274879 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db750623-825d-4af6-befe-785588caef3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:18.275815 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db750623-825d-4af6-befe-785588caef3d admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:18.276151 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db750623-825d-4af6-befe-785588caef3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:18.276473 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db750623-825d-4af6-befe-785588caef3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:18.277263 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db750623-825d-4af6-befe-785588caef3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:18.279264 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db750623-825d-4af6-befe-785588caef3d admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:18.291117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1217/2456] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:11:18.295535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b33411-8c0a-4e6f-814b-32a5d9461cc1 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:18.306394 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b33411-8c0a-4e6f-814b-32a5d9461cc1 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:18.311131 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b65221-e5e6-4620-87e4-0859d12c6872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:18.312011 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b65221-e5e6-4620-87e4-0859d12c6872 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:18.312293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b65221-e5e6-4620-87e4-0859d12c6872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:18.312568 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b65221-e5e6-4620-87e4-0859d12c6872 admin admin] PATH_INFO: `/v3/projects/71c53a043df647fabbb59060e5d581a0/users/9bedac99521e42e1a12674c3c968c55b/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:18.316332 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:18.332452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b65221-e5e6-4620-87e4-0859d12c6872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71c53a043df647fabbb59060e5d581a0, user_id=9bedac99521e42e1a12674c3c968c55b, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:18.337308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b65221-e5e6-4620-87e4-0859d12c6872 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:18.353676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1240/2457] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:11:18 2022] PUT /identity/v3/projects/71c53a043df647fabbb59060e5d581a0/users/9bedac99521e42e1a12674c3c968c55b/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:18.363916 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af7e20-071f-48ba-91b3-356afd838b55 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:18.364231 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af7e20-071f-48ba-91b3-356afd838b55 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:18.364514 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af7e20-071f-48ba-91b3-356afd838b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:18.393225 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-42af7e20-071f-48ba-91b3-356afd838b55 None None] MFA Rules not processed for user `9bedac99521e42e1a12674c3c968c55b`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:18.394507 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-42af7e20-071f-48ba-91b3-356afd838b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:18.459555 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1218/2458] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:11:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:18.464905 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b65221-e5e6-4620-87e4-0859d12c6872 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:18.529631 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b65221-e5e6-4620-87e4-0859d12c6872 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:18.534517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913727a-8558-463f-af8f-da970e82cf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:18.535185 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913727a-8558-463f-af8f-da970e82cf29 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:18.535470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913727a-8558-463f-af8f-da970e82cf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:18.535749 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913727a-8558-463f-af8f-da970e82cf29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:18.536446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913727a-8558-463f-af8f-da970e82cf29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:18.538405 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913727a-8558-463f-af8f-da970e82cf29 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:18.566531 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1241/2459] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:11:18 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:11:18.570776 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af7e20-071f-48ba-91b3-356afd838b55 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:18.584716 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af7e20-071f-48ba-91b3-356afd838b55 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:18.656232 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd970188-39f2-4f31-8c84-f390093479ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:18.656958 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd970188-39f2-4f31-8c84-f390093479ad admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:18.657379 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd970188-39f2-4f31-8c84-f390093479ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:18.657700 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd970188-39f2-4f31-8c84-f390093479ad admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:18.658390 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd970188-39f2-4f31-8c84-f390093479ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:18.660360 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd970188-39f2-4f31-8c84-f390093479ad admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:18.682986 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1219/2460] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:11:18 2022] POST /identity/v3/users => generated 297 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:11:18.686813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913727a-8558-463f-af8f-da970e82cf29 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:18.699340 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913727a-8558-463f-af8f-da970e82cf29 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:18.704329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39810463-7e59-448e-a8ba-746c96474bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:18.705007 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39810463-7e59-448e-a8ba-746c96474bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:18.705294 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39810463-7e59-448e-a8ba-746c96474bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:18.705622 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39810463-7e59-448e-a8ba-746c96474bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:18.706388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39810463-7e59-448e-a8ba-746c96474bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:18.708036 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39810463-7e59-448e-a8ba-746c96474bb0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:18.720176 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1242/2461] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:11:18.724389 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd970188-39f2-4f31-8c84-f390093479ad admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:18.736948 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd970188-39f2-4f31-8c84-f390093479ad admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:18.742131 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5c5a0-af85-46b1-811b-1d33d3b67960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:18.743602 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5c5a0-af85-46b1-811b-1d33d3b67960 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:18.743888 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5c5a0-af85-46b1-811b-1d33d3b67960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:18.744166 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5c5a0-af85-46b1-811b-1d33d3b67960 admin admin] PATH_INFO: `/v3/projects/fb818486d75c45938703e72df21c1435/users/f3281466fafd416485f16d348ba87791/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:18.753576 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:18.761660 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5c5a0-af85-46b1-811b-1d33d3b67960 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb818486d75c45938703e72df21c1435, user_id=f3281466fafd416485f16d348ba87791, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:18.764444 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5c5a0-af85-46b1-811b-1d33d3b67960 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:18.779421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1220/2462] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:11:18 2022] PUT /identity/v3/projects/fb818486d75c45938703e72df21c1435/users/f3281466fafd416485f16d348ba87791/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:18.783619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39810463-7e59-448e-a8ba-746c96474bb0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:18.794833 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39810463-7e59-448e-a8ba-746c96474bb0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:18.860649 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f721f-f54f-4126-9193-4f2da182113f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:18.861411 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f721f-f54f-4126-9193-4f2da182113f admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:18.861697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f721f-f54f-4126-9193-4f2da182113f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:18.861974 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f721f-f54f-4126-9193-4f2da182113f admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:18.862746 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f721f-f54f-4126-9193-4f2da182113f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:18.864366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f721f-f54f-4126-9193-4f2da182113f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:18.879463 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1243/2463] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:11:18.884614 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5c5a0-af85-46b1-811b-1d33d3b67960 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:18.897530 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5c5a0-af85-46b1-811b-1d33d3b67960 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:18.903587 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df0862-005f-411a-8335-34cfb05ef768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:18.904756 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df0862-005f-411a-8335-34cfb05ef768 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:18.905070 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df0862-005f-411a-8335-34cfb05ef768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:18.905355 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df0862-005f-411a-8335-34cfb05ef768 admin admin] PATH_INFO: `/v3/projects/fb818486d75c45938703e72df21c1435/users/f3281466fafd416485f16d348ba87791/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:18.908055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df0862-005f-411a-8335-34cfb05ef768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb818486d75c45938703e72df21c1435, user_id=f3281466fafd416485f16d348ba87791, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:18.910030 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df0862-005f-411a-8335-34cfb05ef768 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:18.921826 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1221/2464] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:11:18 2022] PUT /identity/v3/projects/fb818486d75c45938703e72df21c1435/users/f3281466fafd416485f16d348ba87791/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:18.933426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939f528-744b-4f1d-9c23-c762bd568d2e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:18.933730 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939f528-744b-4f1d-9c23-c762bd568d2e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:18.934016 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939f528-744b-4f1d-9c23-c762bd568d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:18.964599 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d939f528-744b-4f1d-9c23-c762bd568d2e None None] MFA Rules not processed for user `f3281466fafd416485f16d348ba87791`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:18.965785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d939f528-744b-4f1d-9c23-c762bd568d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:19.014801 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:19.045144 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1244/2465] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:11:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:19.057525 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df0862-005f-411a-8335-34cfb05ef768 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:19.059373 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-b6df0862-005f-411a-8335-34cfb05ef768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:19.121634 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df0862-005f-411a-8335-34cfb05ef768 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:19.125442 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c48d7-7077-403f-8828-0a4ede468505 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:19.126091 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c48d7-7077-403f-8828-0a4ede468505 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:19.126412 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c48d7-7077-403f-8828-0a4ede468505 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:19.126696 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c48d7-7077-403f-8828-0a4ede468505 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:19.145650 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c48d7-7077-403f-8828-0a4ede468505 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:19.147156 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c48d7-7077-403f-8828-0a4ede468505 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:19.157986 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1222/2466] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:11:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:19.244840 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939f528-744b-4f1d-9c23-c762bd568d2e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:19.256312 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939f528-744b-4f1d-9c23-c762bd568d2e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:19.266273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a285ca-265c-495e-994c-158eedfb62db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:19.266273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a285ca-265c-495e-994c-158eedfb62db service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:19.266273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a285ca-265c-495e-994c-158eedfb62db service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:19.266273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a285ca-265c-495e-994c-158eedfb62db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:19.323606 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a285ca-265c-495e-994c-158eedfb62db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:19.325558 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a285ca-265c-495e-994c-158eedfb62db service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:19.334574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1245/2467] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:11:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:29.152914 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=36,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:29.330490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=37 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:29.440243 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c48d7-7077-403f-8828-0a4ede468505 service cinder] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:29.513997 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c48d7-7077-403f-8828-0a4ede468505 service cinder] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:29.519198 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d9f4d-e46d-42de-9fa1-a071de37feb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:29.520099 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d9f4d-e46d-42de-9fa1-a071de37feb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:29.520384 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d9f4d-e46d-42de-9fa1-a071de37feb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:29.520662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d9f4d-e46d-42de-9fa1-a071de37feb5 admin admin] PATH_INFO: `/v3/users/9bedac99521e42e1a12674c3c968c55b` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:29.521914 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d9f4d-e46d-42de-9fa1-a071de37feb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bedac99521e42e1a12674c3c968c55b)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:29.524056 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d9f4d-e46d-42de-9fa1-a071de37feb5 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:29.579201 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-200d9f4d-e46d-42de-9fa1-a071de37feb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bedac99521e42e1a12674c3c968c55b'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:29.587446 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-200d9f4d-e46d-42de-9fa1-a071de37feb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bedac99521e42e1a12674c3c968c55b'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:29.588496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-200d9f4d-e46d-42de-9fa1-a071de37feb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bedac99521e42e1a12674c3c968c55b'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:29.589134 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-200d9f4d-e46d-42de-9fa1-a071de37feb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bedac99521e42e1a12674c3c968c55b'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:29.597399 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-200d9f4d-e46d-42de-9fa1-a071de37feb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bedac99521e42e1a12674c3c968c55b'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:29.616938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1223/2468] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:11:29 2022] DELETE /identity/v3/users/9bedac99521e42e1a12674c3c968c55b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:29.622619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de6c3c-d0dd-49cc-8daa-db768bf7a1ef None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:29.622961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de6c3c-d0dd-49cc-8daa-db768bf7a1ef None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:29.623247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de6c3c-d0dd-49cc-8daa-db768bf7a1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:29.643711 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-52de6c3c-d0dd-49cc-8daa-db768bf7a1ef None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:29.645104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-52de6c3c-d0dd-49cc-8daa-db768bf7a1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:29.721672 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1246/2469] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:11:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:11:29.732953 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d9f4d-e46d-42de-9fa1-a071de37feb5 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:29.734434 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-200d9f4d-e46d-42de-9fa1-a071de37feb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:29.803524 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d9f4d-e46d-42de-9fa1-a071de37feb5 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:29.807968 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12d5f3-a71c-4556-bff7-35e4b8a32c95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:29.808628 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12d5f3-a71c-4556-bff7-35e4b8a32c95 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:29.808916 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12d5f3-a71c-4556-bff7-35e4b8a32c95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:29.809194 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12d5f3-a71c-4556-bff7-35e4b8a32c95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:29.818973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12d5f3-a71c-4556-bff7-35e4b8a32c95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:29.820411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12d5f3-a71c-4556-bff7-35e4b8a32c95 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:29.830172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1224/2470] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:11:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:30.125021 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de6c3c-d0dd-49cc-8daa-db768bf7a1ef None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:30.126580 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-52de6c3c-d0dd-49cc-8daa-db768bf7a1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:30.148330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de6c3c-d0dd-49cc-8daa-db768bf7a1ef None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:30.155020 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e06fe-2c46-4543-8208-765f4468d38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:30.156053 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e06fe-2c46-4543-8208-765f4468d38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:30.156486 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e06fe-2c46-4543-8208-765f4468d38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:30.156749 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e06fe-2c46-4543-8208-765f4468d38f admin admin] PATH_INFO: `/v3/projects/71c53a043df647fabbb59060e5d581a0` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:30.157970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e06fe-2c46-4543-8208-765f4468d38f admin admin] RBAC: Authorizing `identity:delete_project(project_id=71c53a043df647fabbb59060e5d581a0)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:30.159973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e06fe-2c46-4543-8208-765f4468d38f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:30.210241 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-752e06fe-2c46-4543-8208-765f4468d38f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71c53a043df647fabbb59060e5d581a0', 'request_id': 'req-752e06fe-2c46-4543-8208-765f4468d38f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:30.217230 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-752e06fe-2c46-4543-8208-765f4468d38f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71c53a043df647fabbb59060e5d581a0', 'request_id': 'req-752e06fe-2c46-4543-8208-765f4468d38f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:30.224163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-752e06fe-2c46-4543-8208-765f4468d38f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71c53a043df647fabbb59060e5d581a0', 'request_id': 'req-752e06fe-2c46-4543-8208-765f4468d38f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:30.239814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-752e06fe-2c46-4543-8208-765f4468d38f admin admin] The token cache is being invalidate because project 71c53a043df647fabbb59060e5d581a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:11:30.240098 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-752e06fe-2c46-4543-8208-765f4468d38f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:30.241260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-752e06fe-2c46-4543-8208-765f4468d38f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:30.245259 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1247/2471] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:11:30 2022] DELETE /identity/v3/projects/71c53a043df647fabbb59060e5d581a0 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:30.248721 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12d5f3-a71c-4556-bff7-35e4b8a32c95 service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:30.259768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12d5f3-a71c-4556-bff7-35e4b8a32c95 service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:30.279365 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:30.323171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ce731-bdbb-4d6f-ab38-9e1307e38553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:30.324078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ce731-bdbb-4d6f-ab38-9e1307e38553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:30.324347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ce731-bdbb-4d6f-ab38-9e1307e38553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:30.324625 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ce731-bdbb-4d6f-ab38-9e1307e38553 admin admin] PATH_INFO: `/v3/users/f3281466fafd416485f16d348ba87791` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:30.325857 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ce731-bdbb-4d6f-ab38-9e1307e38553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3281466fafd416485f16d348ba87791)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:30.327887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ce731-bdbb-4d6f-ab38-9e1307e38553 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:30.375320 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f82ce731-bdbb-4d6f-ab38-9e1307e38553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3281466fafd416485f16d348ba87791'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:30.382640 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f82ce731-bdbb-4d6f-ab38-9e1307e38553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3281466fafd416485f16d348ba87791'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:30.383777 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f82ce731-bdbb-4d6f-ab38-9e1307e38553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3281466fafd416485f16d348ba87791'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:30.384388 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f82ce731-bdbb-4d6f-ab38-9e1307e38553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3281466fafd416485f16d348ba87791'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:30.391774 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f82ce731-bdbb-4d6f-ab38-9e1307e38553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3281466fafd416485f16d348ba87791'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:30.409875 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1225/2472] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:11:30 2022] DELETE /identity/v3/users/f3281466fafd416485f16d348ba87791 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:11:30.725204 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e06fe-2c46-4543-8208-765f4468d38f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:30.726728 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-752e06fe-2c46-4543-8208-765f4468d38f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:30.743331 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e06fe-2c46-4543-8208-765f4468d38f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:30.820144 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e8bc7-595d-4d8b-88e7-7d3a1d9adf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:30.821854 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e8bc7-595d-4d8b-88e7-7d3a1d9adf7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:30.822366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e8bc7-595d-4d8b-88e7-7d3a1d9adf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:30.822648 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e8bc7-595d-4d8b-88e7-7d3a1d9adf7c admin admin] PATH_INFO: `/v3/projects/fb818486d75c45938703e72df21c1435` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:30.824214 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e8bc7-595d-4d8b-88e7-7d3a1d9adf7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb818486d75c45938703e72df21c1435)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:30.826218 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e8bc7-595d-4d8b-88e7-7d3a1d9adf7c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:30.860904 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:30.882483 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-870e8bc7-595d-4d8b-88e7-7d3a1d9adf7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb818486d75c45938703e72df21c1435', 'request_id': 'req-870e8bc7-595d-4d8b-88e7-7d3a1d9adf7c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:30.891504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-870e8bc7-595d-4d8b-88e7-7d3a1d9adf7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb818486d75c45938703e72df21c1435', 'request_id': 'req-870e8bc7-595d-4d8b-88e7-7d3a1d9adf7c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:30.898999 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-870e8bc7-595d-4d8b-88e7-7d3a1d9adf7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb818486d75c45938703e72df21c1435', 'request_id': 'req-870e8bc7-595d-4d8b-88e7-7d3a1d9adf7c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:30.913726 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-870e8bc7-595d-4d8b-88e7-7d3a1d9adf7c admin admin] The token cache is being invalidate because project fb818486d75c45938703e72df21c1435 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:11:30.914018 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-870e8bc7-595d-4d8b-88e7-7d3a1d9adf7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:30.914800 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-870e8bc7-595d-4d8b-88e7-7d3a1d9adf7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:30.916458 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1248/2473] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:11:30 2022] DELETE /identity/v3/projects/fb818486d75c45938703e72df21c1435 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:30.921501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae9a70-fe5a-497f-a41d-c58dab0c115b None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:30.921788 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae9a70-fe5a-497f-a41d-c58dab0c115b None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:30.922063 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae9a70-fe5a-497f-a41d-c58dab0c115b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:30.948619 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-59ae9a70-fe5a-497f-a41d-c58dab0c115b None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:30.949902 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-59ae9a70-fe5a-497f-a41d-c58dab0c115b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:31.021201 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1226/2474] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:11:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:31.032991 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92faa33d-0211-4f4c-aa67-fa74259f2932 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:31.033276 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92faa33d-0211-4f4c-aa67-fa74259f2932 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:31.033572 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92faa33d-0211-4f4c-aa67-fa74259f2932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:31.053375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-92faa33d-0211-4f4c-aa67-fa74259f2932 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:31.054645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-92faa33d-0211-4f4c-aa67-fa74259f2932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:31.063317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1249/2475] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:11:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:31.067689 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae9a70-fe5a-497f-a41d-c58dab0c115b None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:31.078514 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae9a70-fe5a-497f-a41d-c58dab0c115b None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:31.083822 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df533f1-127d-4cb3-8cb0-3ecc78c5057a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:31.084498 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df533f1-127d-4cb3-8cb0-3ecc78c5057a admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:31.084782 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df533f1-127d-4cb3-8cb0-3ecc78c5057a admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:31.085059 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df533f1-127d-4cb3-8cb0-3ecc78c5057a admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:31.085795 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df533f1-127d-4cb3-8cb0-3ecc78c5057a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:11:31.086103 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df533f1-127d-4cb3-8cb0-3ecc78c5057a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:31.088207 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df533f1-127d-4cb3-8cb0-3ecc78c5057a admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:31.100644 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1227/2476] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:11:31.106290 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92faa33d-0211-4f4c-aa67-fa74259f2932 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:31.115933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92faa33d-0211-4f4c-aa67-fa74259f2932 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:31.121534 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747dfa46-de7a-4312-98ab-36883685983e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:31.122226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dfa46-de7a-4312-98ab-36883685983e admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:31.122515 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dfa46-de7a-4312-98ab-36883685983e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:31.122803 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dfa46-de7a-4312-98ab-36883685983e admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:31.123507 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747dfa46-de7a-4312-98ab-36883685983e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:31.125431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747dfa46-de7a-4312-98ab-36883685983e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:31.155004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1250/2477] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:11:31 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:11:31.159252 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df533f1-127d-4cb3-8cb0-3ecc78c5057a admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:31.170643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df533f1-127d-4cb3-8cb0-3ecc78c5057a admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:31.198864 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:31.234710 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e865d-b3df-4289-a577-c240a7b8a4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:31.235307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e865d-b3df-4289-a577-c240a7b8a4f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:31.235624 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e865d-b3df-4289-a577-c240a7b8a4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:31.235943 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e865d-b3df-4289-a577-c240a7b8a4f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:31.236608 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e865d-b3df-4289-a577-c240a7b8a4f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:31.238698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e865d-b3df-4289-a577-c240a7b8a4f3 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:31.260917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1228/2478] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:11:31 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:11:31.266068 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747dfa46-de7a-4312-98ab-36883685983e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:31.277013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747dfa46-de7a-4312-98ab-36883685983e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:31.282022 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fa59a-d225-44bc-8330-e4022293505a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:31.282749 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fa59a-d225-44bc-8330-e4022293505a admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:31.283038 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fa59a-d225-44bc-8330-e4022293505a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:31.283327 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fa59a-d225-44bc-8330-e4022293505a admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:31.284087 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fa59a-d225-44bc-8330-e4022293505a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:31.285750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fa59a-d225-44bc-8330-e4022293505a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:31.297566 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1251/2479] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:11:31.301747 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e865d-b3df-4289-a577-c240a7b8a4f3 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:31.312935 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e865d-b3df-4289-a577-c240a7b8a4f3 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:31.317654 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f849f4-7b6f-4d30-8b1d-910fe2d2de9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:31.318358 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f849f4-7b6f-4d30-8b1d-910fe2d2de9c admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:31.318703 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f849f4-7b6f-4d30-8b1d-910fe2d2de9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:31.318992 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f849f4-7b6f-4d30-8b1d-910fe2d2de9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:31.319725 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f849f4-7b6f-4d30-8b1d-910fe2d2de9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:31.321423 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f849f4-7b6f-4d30-8b1d-910fe2d2de9c admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:31.332837 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1229/2480] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:11:31.337676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fa59a-d225-44bc-8330-e4022293505a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:31.347986 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fa59a-d225-44bc-8330-e4022293505a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:31.353184 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010bf000-75de-45d8-9ceb-6bdd9733d0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:31.354118 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bf000-75de-45d8-9ceb-6bdd9733d0f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:31.354439 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bf000-75de-45d8-9ceb-6bdd9733d0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:31.354719 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bf000-75de-45d8-9ceb-6bdd9733d0f0 admin admin] PATH_INFO: `/v3/projects/79e89cb675a74edbbe6ea25f04fec177/users/b9787b22f05b4b05936698e9b418db67/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:31.370844 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010bf000-75de-45d8-9ceb-6bdd9733d0f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79e89cb675a74edbbe6ea25f04fec177, user_id=b9787b22f05b4b05936698e9b418db67, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:31.373561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010bf000-75de-45d8-9ceb-6bdd9733d0f0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:31.392736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1252/2481] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:11:31 2022] PUT /identity/v3/projects/79e89cb675a74edbbe6ea25f04fec177/users/b9787b22f05b4b05936698e9b418db67/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:11:31.404430 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616db9f-d3db-4824-8934-8916d7ff8a36 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:31.404724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616db9f-d3db-4824-8934-8916d7ff8a36 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:31.405073 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616db9f-d3db-4824-8934-8916d7ff8a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:31.435515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-8616db9f-d3db-4824-8934-8916d7ff8a36 None None] MFA Rules not processed for user `b9787b22f05b4b05936698e9b418db67`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:31.436851 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-8616db9f-d3db-4824-8934-8916d7ff8a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:31.508077 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1230/2482] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:11:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:31.514701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010bf000-75de-45d8-9ceb-6bdd9733d0f0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:31.585854 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010bf000-75de-45d8-9ceb-6bdd9733d0f0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:31.591576 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76aebec-e8ec-4b02-972e-b7efc176c4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:31.592245 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76aebec-e8ec-4b02-972e-b7efc176c4ea admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:31.592529 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76aebec-e8ec-4b02-972e-b7efc176c4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:31.592807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76aebec-e8ec-4b02-972e-b7efc176c4ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:31.593510 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76aebec-e8ec-4b02-972e-b7efc176c4ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:31.595517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76aebec-e8ec-4b02-972e-b7efc176c4ea admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:31.623224 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1253/2483] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:11:31 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:11:31.627048 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616db9f-d3db-4824-8934-8916d7ff8a36 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:31.643403 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616db9f-d3db-4824-8934-8916d7ff8a36 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:31.702543 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df544777-7db9-4a0b-a7e2-943f90112fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:31.703259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df544777-7db9-4a0b-a7e2-943f90112fb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:31.703548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df544777-7db9-4a0b-a7e2-943f90112fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:31.703854 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df544777-7db9-4a0b-a7e2-943f90112fb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:31.704568 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df544777-7db9-4a0b-a7e2-943f90112fb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:31.706536 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df544777-7db9-4a0b-a7e2-943f90112fb7 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:31.729043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1231/2484] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:11:31 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:11:31.733458 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76aebec-e8ec-4b02-972e-b7efc176c4ea admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:31.744175 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76aebec-e8ec-4b02-972e-b7efc176c4ea admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:31.748956 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411d763-b670-4641-9a2c-e469cbceea57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:31.749629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411d763-b670-4641-9a2c-e469cbceea57 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:31.749916 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411d763-b670-4641-9a2c-e469cbceea57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:31.750215 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411d763-b670-4641-9a2c-e469cbceea57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:31.750953 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6411d763-b670-4641-9a2c-e469cbceea57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:31.752580 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6411d763-b670-4641-9a2c-e469cbceea57 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:31.764978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1254/2485] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:11:31.769223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df544777-7db9-4a0b-a7e2-943f90112fb7 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:31.780493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df544777-7db9-4a0b-a7e2-943f90112fb7 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:31.785157 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8e09d-e56c-42ac-b244-547b88fc7275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:31.786046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8e09d-e56c-42ac-b244-547b88fc7275 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:31.786354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8e09d-e56c-42ac-b244-547b88fc7275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:31.786633 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8e09d-e56c-42ac-b244-547b88fc7275 admin admin] PATH_INFO: `/v3/projects/c6b05bb95d1d46bcb83cd212badc4ed3/users/34f613e2b2c34bf4be54d4d8a5c9318e/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:31.804765 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a8e09d-e56c-42ac-b244-547b88fc7275 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6b05bb95d1d46bcb83cd212badc4ed3, user_id=34f613e2b2c34bf4be54d4d8a5c9318e, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:31.809284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a8e09d-e56c-42ac-b244-547b88fc7275 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:31.827032 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1232/2486] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:11:31 2022] PUT /identity/v3/projects/c6b05bb95d1d46bcb83cd212badc4ed3/users/34f613e2b2c34bf4be54d4d8a5c9318e/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:31.831275 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411d763-b670-4641-9a2c-e469cbceea57 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:31.844122 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411d763-b670-4641-9a2c-e469cbceea57 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:31.873928 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:31.916406 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbacc5-bb13-449f-8d43-92fcde6095db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:31.920774 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbacc5-bb13-449f-8d43-92fcde6095db admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:31.920774 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbacc5-bb13-449f-8d43-92fcde6095db admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:31.920774 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbacc5-bb13-449f-8d43-92fcde6095db admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:31.920774 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bbacc5-bb13-449f-8d43-92fcde6095db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:31.920774 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bbacc5-bb13-449f-8d43-92fcde6095db admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:31.934461 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1255/2487] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:11:31.938694 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8e09d-e56c-42ac-b244-547b88fc7275 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:31.944408 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:31.952280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8e09d-e56c-42ac-b244-547b88fc7275 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:31.958707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9fb3b-ec57-4285-986d-6d99475ccac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:31.959614 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9fb3b-ec57-4285-986d-6d99475ccac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:31.959903 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9fb3b-ec57-4285-986d-6d99475ccac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:31.960185 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9fb3b-ec57-4285-986d-6d99475ccac3 admin admin] PATH_INFO: `/v3/projects/c6b05bb95d1d46bcb83cd212badc4ed3/users/34f613e2b2c34bf4be54d4d8a5c9318e/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:31.962877 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9fb3b-ec57-4285-986d-6d99475ccac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6b05bb95d1d46bcb83cd212badc4ed3, user_id=34f613e2b2c34bf4be54d4d8a5c9318e, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:31.965537 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9fb3b-ec57-4285-986d-6d99475ccac3 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:31.976900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1233/2488] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:11:31 2022] PUT /identity/v3/projects/c6b05bb95d1d46bcb83cd212badc4ed3/users/34f613e2b2c34bf4be54d4d8a5c9318e/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:31.988572 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0835b3-5d99-4df9-add8-e5095b0a0ead None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:31.988868 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0835b3-5d99-4df9-add8-e5095b0a0ead None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:31.989154 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0835b3-5d99-4df9-add8-e5095b0a0ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:32.019729 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1f0835b3-5d99-4df9-add8-e5095b0a0ead None None] MFA Rules not processed for user `34f613e2b2c34bf4be54d4d8a5c9318e`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:32.021271 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1f0835b3-5d99-4df9-add8-e5095b0a0ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:32.099363 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1256/2489] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:11:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:32.110861 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9fb3b-ec57-4285-986d-6d99475ccac3 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:32.112909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-12f9fb3b-ec57-4285-986d-6d99475ccac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:32.188847 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9fb3b-ec57-4285-986d-6d99475ccac3 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:32.192647 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb835810-417a-4e67-ab67-3d4385ad11c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:32.193304 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb835810-417a-4e67-ab67-3d4385ad11c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:32.193598 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb835810-417a-4e67-ab67-3d4385ad11c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:32.193855 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb835810-417a-4e67-ab67-3d4385ad11c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:32.254914 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb835810-417a-4e67-ab67-3d4385ad11c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:32.256385 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb835810-417a-4e67-ab67-3d4385ad11c6 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:32.266233 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1234/2490] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:11:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:36.869443 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0835b3-5d99-4df9-add8-e5095b0a0ead None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:36.870877 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1f0835b3-5d99-4df9-add8-e5095b0a0ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:36.882766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0835b3-5d99-4df9-add8-e5095b0a0ead None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:36.886530 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46463eb-77c5-42b2-a4b0-0fc9a0364af1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:36.887176 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46463eb-77c5-42b2-a4b0-0fc9a0364af1 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:36.887490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46463eb-77c5-42b2-a4b0-0fc9a0364af1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:36.887769 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46463eb-77c5-42b2-a4b0-0fc9a0364af1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:36.896783 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46463eb-77c5-42b2-a4b0-0fc9a0364af1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:36.898233 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46463eb-77c5-42b2-a4b0-0fc9a0364af1 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:36.908917 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1257/2491] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:11:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:42.262060 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:46.904574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=58 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:51.647797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb835810-417a-4e67-ab67-3d4385ad11c6 service cinder] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:51.651072 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-cb835810-417a-4e67-ab67-3d4385ad11c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:51.714158 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb835810-417a-4e67-ab67-3d4385ad11c6 service cinder] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:51.718445 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b6822f-7b78-413e-ba1f-8a2e4220f654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:51.719137 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6822f-7b78-413e-ba1f-8a2e4220f654 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:51.719424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6822f-7b78-413e-ba1f-8a2e4220f654 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:51.719702 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6822f-7b78-413e-ba1f-8a2e4220f654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:51.721501 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-86b6822f-7b78-413e-ba1f-8a2e4220f654 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:51.794957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b6822f-7b78-413e-ba1f-8a2e4220f654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:51.796869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b6822f-7b78-413e-ba1f-8a2e4220f654 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:51.809732 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1235/2492] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:11:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:11:55.033042 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46463eb-77c5-42b2-a4b0-0fc9a0364af1 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:55.034756 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f46463eb-77c5-42b2-a4b0-0fc9a0364af1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:55.050744 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46463eb-77c5-42b2-a4b0-0fc9a0364af1 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:55.057295 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea7b07-2fff-4f88-8f30-8f722bc1805f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:55.058413 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea7b07-2fff-4f88-8f30-8f722bc1805f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:55.058703 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea7b07-2fff-4f88-8f30-8f722bc1805f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:55.058984 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea7b07-2fff-4f88-8f30-8f722bc1805f admin admin] PATH_INFO: `/v3/users/f6b5113ea0f4484ba91a5f0e6184ca35` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:55.060185 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea7b07-2fff-4f88-8f30-8f722bc1805f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6b5113ea0f4484ba91a5f0e6184ca35)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:55.062106 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea7b07-2fff-4f88-8f30-8f722bc1805f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:55.118215 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-41ea7b07-2fff-4f88-8f30-8f722bc1805f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6b5113ea0f4484ba91a5f0e6184ca35'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:55.126291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-41ea7b07-2fff-4f88-8f30-8f722bc1805f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6b5113ea0f4484ba91a5f0e6184ca35'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:55.145078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-41ea7b07-2fff-4f88-8f30-8f722bc1805f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6b5113ea0f4484ba91a5f0e6184ca35'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:55.145801 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-41ea7b07-2fff-4f88-8f30-8f722bc1805f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6b5113ea0f4484ba91a5f0e6184ca35'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:55.146506 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-41ea7b07-2fff-4f88-8f30-8f722bc1805f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6b5113ea0f4484ba91a5f0e6184ca35'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:55.160308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1258/2493] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:11:55 2022] DELETE /identity/v3/users/f6b5113ea0f4484ba91a5f0e6184ca35 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:55.343950 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b6822f-7b78-413e-ba1f-8a2e4220f654 service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:55.354676 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b6822f-7b78-413e-ba1f-8a2e4220f654 service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:55.415409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911536d-147d-4176-843e-1226c13b4dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:55.416278 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911536d-147d-4176-843e-1226c13b4dd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:55.416564 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911536d-147d-4176-843e-1226c13b4dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:55.416841 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911536d-147d-4176-843e-1226c13b4dd6 admin admin] PATH_INFO: `/v3/projects/c0b976166b884524b9a3855712be72de` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:55.418013 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911536d-147d-4176-843e-1226c13b4dd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0b976166b884524b9a3855712be72de)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:55.419948 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911536d-147d-4176-843e-1226c13b4dd6 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:55.467442 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-5911536d-147d-4176-843e-1226c13b4dd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0b976166b884524b9a3855712be72de', 'request_id': 'req-5911536d-147d-4176-843e-1226c13b4dd6'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:55.470026 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:55.475564 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-5911536d-147d-4176-843e-1226c13b4dd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0b976166b884524b9a3855712be72de', 'request_id': 'req-5911536d-147d-4176-843e-1226c13b4dd6'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:55.483505 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-5911536d-147d-4176-843e-1226c13b4dd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0b976166b884524b9a3855712be72de', 'request_id': 'req-5911536d-147d-4176-843e-1226c13b4dd6'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:55.494758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-5911536d-147d-4176-843e-1226c13b4dd6 admin admin] The token cache is being invalidate because project c0b976166b884524b9a3855712be72de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:11:55.495046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-5911536d-147d-4176-843e-1226c13b4dd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:55.495766 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-5911536d-147d-4176-843e-1226c13b4dd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:11:55.497390 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1236/2494] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:11:55 2022] DELETE /identity/v3/projects/c0b976166b884524b9a3855712be72de => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:55.502664 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5381a2-a987-4ccb-b818-e7a3f8e1156d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:55.503190 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5381a2-a987-4ccb-b818-e7a3f8e1156d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:55.503476 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5381a2-a987-4ccb-b818-e7a3f8e1156d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:55.522264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ba5381a2-a987-4ccb-b818-e7a3f8e1156d None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:55.523927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ba5381a2-a987-4ccb-b818-e7a3f8e1156d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:55.585099 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1259/2495] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:11:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:55.593701 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b78e46-b99b-43e5-958a-92f3946296cf None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:55.593994 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b78e46-b99b-43e5-958a-92f3946296cf None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:55.594386 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b78e46-b99b-43e5-958a-92f3946296cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:55.615232 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-a0b78e46-b99b-43e5-958a-92f3946296cf None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:55.616448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-a0b78e46-b99b-43e5-958a-92f3946296cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:55.624782 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1237/2496] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:11:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:55.629160 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5381a2-a987-4ccb-b818-e7a3f8e1156d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:55.639200 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5381a2-a987-4ccb-b818-e7a3f8e1156d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:55.644092 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbad03-d6ba-4c44-bc68-00544aebcb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:55.644747 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbad03-d6ba-4c44-bc68-00544aebcb35 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:55.645032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbad03-d6ba-4c44-bc68-00544aebcb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:55.645314 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbad03-d6ba-4c44-bc68-00544aebcb35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:55.646051 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addbad03-d6ba-4c44-bc68-00544aebcb35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:11:55.646396 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addbad03-d6ba-4c44-bc68-00544aebcb35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:55.648003 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addbad03-d6ba-4c44-bc68-00544aebcb35 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:55.661356 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1260/2497] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:11:55.666763 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b78e46-b99b-43e5-958a-92f3946296cf None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:55.675836 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b78e46-b99b-43e5-958a-92f3946296cf None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:55.681074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618afcb6-9d70-4fba-9f46-cb497e62b7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:55.681726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618afcb6-9d70-4fba-9f46-cb497e62b7a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:55.682008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618afcb6-9d70-4fba-9f46-cb497e62b7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:55.682315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618afcb6-9d70-4fba-9f46-cb497e62b7a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:55.683005 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618afcb6-9d70-4fba-9f46-cb497e62b7a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:55.684919 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618afcb6-9d70-4fba-9f46-cb497e62b7a5 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:55.710057 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1238/2498] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:11:55 2022] POST /identity/v3/projects => generated 367 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:11:55.715080 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbad03-d6ba-4c44-bc68-00544aebcb35 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:55.725457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbad03-d6ba-4c44-bc68-00544aebcb35 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:55.782999 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb4ed0-4ed0-49d1-9074-3787816c2684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:55.783563 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb4ed0-4ed0-49d1-9074-3787816c2684 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:55.783853 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb4ed0-4ed0-49d1-9074-3787816c2684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:55.784130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb4ed0-4ed0-49d1-9074-3787816c2684 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:55.784836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb4ed0-4ed0-49d1-9074-3787816c2684 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:55.786801 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb4ed0-4ed0-49d1-9074-3787816c2684 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:55.794557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:11:55.809658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1261/2499] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:11:55 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:11:55.813850 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618afcb6-9d70-4fba-9f46-cb497e62b7a5 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:55.824667 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618afcb6-9d70-4fba-9f46-cb497e62b7a5 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:55.829290 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d227dc-ffce-4aa4-a922-13eaa31d217b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:55.829945 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d227dc-ffce-4aa4-a922-13eaa31d217b admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:55.830263 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d227dc-ffce-4aa4-a922-13eaa31d217b admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:55.830542 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d227dc-ffce-4aa4-a922-13eaa31d217b admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:55.831255 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d227dc-ffce-4aa4-a922-13eaa31d217b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:55.832871 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d227dc-ffce-4aa4-a922-13eaa31d217b admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:55.844264 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1239/2500] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:11:55.848506 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb4ed0-4ed0-49d1-9074-3787816c2684 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:55.858811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb4ed0-4ed0-49d1-9074-3787816c2684 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:55.863710 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5700ef-0e79-4387-9158-53a2a2e5d62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:55.864315 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5700ef-0e79-4387-9158-53a2a2e5d62a admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:55.864598 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5700ef-0e79-4387-9158-53a2a2e5d62a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:55.864873 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5700ef-0e79-4387-9158-53a2a2e5d62a admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:55.865617 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5700ef-0e79-4387-9158-53a2a2e5d62a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:55.867251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5700ef-0e79-4387-9158-53a2a2e5d62a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:55.877235 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1262/2501] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:11:55.881441 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d227dc-ffce-4aa4-a922-13eaa31d217b admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:55.892254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d227dc-ffce-4aa4-a922-13eaa31d217b admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:55.896891 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bce073-49a6-43fe-bd41-3ed8862c7538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:55.897767 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bce073-49a6-43fe-bd41-3ed8862c7538 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:55.898048 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bce073-49a6-43fe-bd41-3ed8862c7538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:55.899083 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bce073-49a6-43fe-bd41-3ed8862c7538 admin admin] PATH_INFO: `/v3/projects/ea248afbc39f4c4ba9dbcbfb03b46731/users/c03d2a22086a41138bf038ebab42e0fd/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:55.914653 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bce073-49a6-43fe-bd41-3ed8862c7538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea248afbc39f4c4ba9dbcbfb03b46731, user_id=c03d2a22086a41138bf038ebab42e0fd, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:55.917353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bce073-49a6-43fe-bd41-3ed8862c7538 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:55.930922 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1240/2502] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:11:55 2022] PUT /identity/v3/projects/ea248afbc39f4c4ba9dbcbfb03b46731/users/c03d2a22086a41138bf038ebab42e0fd/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:11:55.942530 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b99de1-b4f1-4a8b-a7d4-85e70ee6769d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:55.942877 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b99de1-b4f1-4a8b-a7d4-85e70ee6769d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:55.943158 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b99de1-b4f1-4a8b-a7d4-85e70ee6769d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:55.969371 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c5b99de1-b4f1-4a8b-a7d4-85e70ee6769d None None] MFA Rules not processed for user `c03d2a22086a41138bf038ebab42e0fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:11:55.970615 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c5b99de1-b4f1-4a8b-a7d4-85e70ee6769d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:56.037643 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1263/2503] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:11:55 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:11:56.049421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bce073-49a6-43fe-bd41-3ed8862c7538 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:11:56.051395 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-c0bce073-49a6-43fe-bd41-3ed8862c7538 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:11:56.109050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bce073-49a6-43fe-bd41-3ed8862c7538 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:11:56.112732 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72135642-c5ea-431e-aeef-4f200b42d472 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:11:56.113375 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72135642-c5ea-431e-aeef-4f200b42d472 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:11:56.113661 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72135642-c5ea-431e-aeef-4f200b42d472 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:11:56.113953 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72135642-c5ea-431e-aeef-4f200b42d472 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:11:56.123015 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72135642-c5ea-431e-aeef-4f200b42d472 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:11:56.124443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72135642-c5ea-431e-aeef-4f200b42d472 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:11:56.133369 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1241/2504] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:11:56 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:00.866666 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b99de1-b4f1-4a8b-a7d4-85e70ee6769d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:00.868856 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c5b99de1-b4f1-4a8b-a7d4-85e70ee6769d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:00.944283 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b99de1-b4f1-4a8b-a7d4-85e70ee6769d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:00.949567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0fc99a-0cfd-4bc9-b487-72a8e9988712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:00.950602 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fc99a-0cfd-4bc9-b487-72a8e9988712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:00.950883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fc99a-0cfd-4bc9-b487-72a8e9988712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:00.951159 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fc99a-0cfd-4bc9-b487-72a8e9988712 admin admin] PATH_INFO: `/v3/users/b9787b22f05b4b05936698e9b418db67` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:00.952352 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0fc99a-0cfd-4bc9-b487-72a8e9988712 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9787b22f05b4b05936698e9b418db67)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:00.954309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0fc99a-0cfd-4bc9-b487-72a8e9988712 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:01.009965 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cd0fc99a-0cfd-4bc9-b487-72a8e9988712 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9787b22f05b4b05936698e9b418db67'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:01.018377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cd0fc99a-0cfd-4bc9-b487-72a8e9988712 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9787b22f05b4b05936698e9b418db67'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:01.030108 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:01.031422 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cd0fc99a-0cfd-4bc9-b487-72a8e9988712 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9787b22f05b4b05936698e9b418db67'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:01.032173 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cd0fc99a-0cfd-4bc9-b487-72a8e9988712 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9787b22f05b4b05936698e9b418db67'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:01.032937 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cd0fc99a-0cfd-4bc9-b487-72a8e9988712 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9787b22f05b4b05936698e9b418db67'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:01.048611 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1264/2505] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:12:00 2022] DELETE /identity/v3/users/b9787b22f05b4b05936698e9b418db67 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:01.053209 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce974e21-6682-42e6-a27e-87f2ff844c28 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:01.053723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce974e21-6682-42e6-a27e-87f2ff844c28 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:01.053893 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce974e21-6682-42e6-a27e-87f2ff844c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:01.072729 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-ce974e21-6682-42e6-a27e-87f2ff844c28 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:01.073947 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-ce974e21-6682-42e6-a27e-87f2ff844c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:01.142696 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1242/2506] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:12:01.152504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0fc99a-0cfd-4bc9-b487-72a8e9988712 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:01.154363 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cd0fc99a-0cfd-4bc9-b487-72a8e9988712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:01.220888 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0fc99a-0cfd-4bc9-b487-72a8e9988712 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:01.224535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b90096-6012-45a0-a273-cea30e684d51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:01.225181 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b90096-6012-45a0-a273-cea30e684d51 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:01.225466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b90096-6012-45a0-a273-cea30e684d51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:01.225775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b90096-6012-45a0-a273-cea30e684d51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:01.235463 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b90096-6012-45a0-a273-cea30e684d51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:01.236891 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b90096-6012-45a0-a273-cea30e684d51 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:01.246771 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1265/2507] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:01.531088 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce974e21-6682-42e6-a27e-87f2ff844c28 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:01.532787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-ce974e21-6682-42e6-a27e-87f2ff844c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:01.546900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce974e21-6682-42e6-a27e-87f2ff844c28 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:01.551875 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f476ff-6e90-4538-8a06-6a5d7c25b9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:01.552864 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f476ff-6e90-4538-8a06-6a5d7c25b9ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:01.553334 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f476ff-6e90-4538-8a06-6a5d7c25b9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:01.553556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f476ff-6e90-4538-8a06-6a5d7c25b9ec admin admin] PATH_INFO: `/v3/projects/79e89cb675a74edbbe6ea25f04fec177` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:01.556065 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f476ff-6e90-4538-8a06-6a5d7c25b9ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=79e89cb675a74edbbe6ea25f04fec177)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:01.558291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f476ff-6e90-4538-8a06-6a5d7c25b9ec admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:01.576200 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:01.608186 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-60f476ff-6e90-4538-8a06-6a5d7c25b9ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79e89cb675a74edbbe6ea25f04fec177', 'request_id': 'req-60f476ff-6e90-4538-8a06-6a5d7c25b9ec'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:01.615683 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-60f476ff-6e90-4538-8a06-6a5d7c25b9ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79e89cb675a74edbbe6ea25f04fec177', 'request_id': 'req-60f476ff-6e90-4538-8a06-6a5d7c25b9ec'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:01.622007 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-60f476ff-6e90-4538-8a06-6a5d7c25b9ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79e89cb675a74edbbe6ea25f04fec177', 'request_id': 'req-60f476ff-6e90-4538-8a06-6a5d7c25b9ec'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:01.633808 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-60f476ff-6e90-4538-8a06-6a5d7c25b9ec admin admin] The token cache is being invalidate because project 79e89cb675a74edbbe6ea25f04fec177 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:01.634095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-60f476ff-6e90-4538-8a06-6a5d7c25b9ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:01.634773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-60f476ff-6e90-4538-8a06-6a5d7c25b9ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:01.636320 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1243/2508] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:12:01 2022] DELETE /identity/v3/projects/79e89cb675a74edbbe6ea25f04fec177 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:01.640297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b90096-6012-45a0-a273-cea30e684d51 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:01.649916 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b90096-6012-45a0-a273-cea30e684d51 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:01.707786 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7302071-96b0-4b2e-824d-1e916c747693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:01.708649 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7302071-96b0-4b2e-824d-1e916c747693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:01.708935 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7302071-96b0-4b2e-824d-1e916c747693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:01.709213 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7302071-96b0-4b2e-824d-1e916c747693 admin admin] PATH_INFO: `/v3/users/34f613e2b2c34bf4be54d4d8a5c9318e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:01.710419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7302071-96b0-4b2e-824d-1e916c747693 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34f613e2b2c34bf4be54d4d8a5c9318e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:01.712352 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7302071-96b0-4b2e-824d-1e916c747693 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:01.757181 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7302071-96b0-4b2e-824d-1e916c747693 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34f613e2b2c34bf4be54d4d8a5c9318e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:01.765115 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7302071-96b0-4b2e-824d-1e916c747693 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34f613e2b2c34bf4be54d4d8a5c9318e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:01.777294 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7302071-96b0-4b2e-824d-1e916c747693 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34f613e2b2c34bf4be54d4d8a5c9318e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:01.778006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7302071-96b0-4b2e-824d-1e916c747693 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34f613e2b2c34bf4be54d4d8a5c9318e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:01.778924 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7302071-96b0-4b2e-824d-1e916c747693 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34f613e2b2c34bf4be54d4d8a5c9318e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:01.791482 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1266/2509] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:12:01 2022] DELETE /identity/v3/users/34f613e2b2c34bf4be54d4d8a5c9318e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:02.071174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f476ff-6e90-4538-8a06-6a5d7c25b9ec admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:02.073006 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-60f476ff-6e90-4538-8a06-6a5d7c25b9ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:02.087342 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f476ff-6e90-4538-8a06-6a5d7c25b9ec admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:02.145861 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87fff7-5e0e-4f90-89de-97a9e9112e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:02.146775 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87fff7-5e0e-4f90-89de-97a9e9112e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:02.147065 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87fff7-5e0e-4f90-89de-97a9e9112e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:02.147344 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87fff7-5e0e-4f90-89de-97a9e9112e3a admin admin] PATH_INFO: `/v3/projects/c6b05bb95d1d46bcb83cd212badc4ed3` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:02.149269 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87fff7-5e0e-4f90-89de-97a9e9112e3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6b05bb95d1d46bcb83cd212badc4ed3)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:02.151435 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87fff7-5e0e-4f90-89de-97a9e9112e3a admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:02.199005 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-eb87fff7-5e0e-4f90-89de-97a9e9112e3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6b05bb95d1d46bcb83cd212badc4ed3', 'request_id': 'req-eb87fff7-5e0e-4f90-89de-97a9e9112e3a'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:02.206593 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-eb87fff7-5e0e-4f90-89de-97a9e9112e3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6b05bb95d1d46bcb83cd212badc4ed3', 'request_id': 'req-eb87fff7-5e0e-4f90-89de-97a9e9112e3a'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:02.212826 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-eb87fff7-5e0e-4f90-89de-97a9e9112e3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6b05bb95d1d46bcb83cd212badc4ed3', 'request_id': 'req-eb87fff7-5e0e-4f90-89de-97a9e9112e3a'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:02.225487 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-eb87fff7-5e0e-4f90-89de-97a9e9112e3a admin admin] The token cache is being invalidate because project c6b05bb95d1d46bcb83cd212badc4ed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:02.225774 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-eb87fff7-5e0e-4f90-89de-97a9e9112e3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:02.227244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-eb87fff7-5e0e-4f90-89de-97a9e9112e3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:02.228759 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1244/2510] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:12:02 2022] DELETE /identity/v3/projects/c6b05bb95d1d46bcb83cd212badc4ed3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:02.233808 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02346512-16a2-47ce-9df4-4d861e99e42d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:02.234332 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02346512-16a2-47ce-9df4-4d861e99e42d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:02.234453 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02346512-16a2-47ce-9df4-4d861e99e42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:02.258339 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-02346512-16a2-47ce-9df4-4d861e99e42d None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:02.259756 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-02346512-16a2-47ce-9df4-4d861e99e42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:02.264300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:02.332460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1267/2511] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:02.341361 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b7d83-01f4-4c78-8b1c-f8683fb315a6 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:02.341656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b7d83-01f4-4c78-8b1c-f8683fb315a6 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:02.341943 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b7d83-01f4-4c78-8b1c-f8683fb315a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:02.361253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-570b7d83-01f4-4c78-8b1c-f8683fb315a6 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:02.362494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-570b7d83-01f4-4c78-8b1c-f8683fb315a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:02.371566 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1245/2512] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:02.376490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02346512-16a2-47ce-9df4-4d861e99e42d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:02.387878 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02346512-16a2-47ce-9df4-4d861e99e42d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:02.393719 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd98a7b-1bfe-437b-bb6c-1485fa01fb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:02.394447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd98a7b-1bfe-437b-bb6c-1485fa01fb2c admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:02.394756 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd98a7b-1bfe-437b-bb6c-1485fa01fb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:02.395038 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd98a7b-1bfe-437b-bb6c-1485fa01fb2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:02.395814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd98a7b-1bfe-437b-bb6c-1485fa01fb2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:12:02.396162 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd98a7b-1bfe-437b-bb6c-1485fa01fb2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:02.397879 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd98a7b-1bfe-437b-bb6c-1485fa01fb2c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:02.410893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1268/2513] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:12:02.415308 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b7d83-01f4-4c78-8b1c-f8683fb315a6 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:02.424545 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b7d83-01f4-4c78-8b1c-f8683fb315a6 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:02.429284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb47384-1206-4502-b00b-0f98e0fbddf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:02.430654 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb47384-1206-4502-b00b-0f98e0fbddf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:02.430948 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb47384-1206-4502-b00b-0f98e0fbddf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:02.431256 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb47384-1206-4502-b00b-0f98e0fbddf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:02.431973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb47384-1206-4502-b00b-0f98e0fbddf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:02.433880 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb47384-1206-4502-b00b-0f98e0fbddf4 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:02.459557 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1246/2514] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:12:02 2022] POST /identity/v3/projects => generated 409 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:02.463742 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd98a7b-1bfe-437b-bb6c-1485fa01fb2c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:02.474324 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd98a7b-1bfe-437b-bb6c-1485fa01fb2c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:02.533680 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1a4ee-c2a6-46c9-98c0-1f98c2379729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:02.534383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1a4ee-c2a6-46c9-98c0-1f98c2379729 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:02.534712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1a4ee-c2a6-46c9-98c0-1f98c2379729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:02.535031 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1a4ee-c2a6-46c9-98c0-1f98c2379729 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:02.535782 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a1a4ee-c2a6-46c9-98c0-1f98c2379729 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:02.537769 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a1a4ee-c2a6-46c9-98c0-1f98c2379729 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:02.559195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1269/2515] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:12:02 2022] POST /identity/v3/users => generated 305 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:02.563432 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb47384-1206-4502-b00b-0f98e0fbddf4 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:02.570753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:02.575142 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb47384-1206-4502-b00b-0f98e0fbddf4 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:02.581462 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6b2c2-a55a-4915-86eb-05a7b22e90cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:02.582215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6b2c2-a55a-4915-86eb-05a7b22e90cc admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:02.582665 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6b2c2-a55a-4915-86eb-05a7b22e90cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:02.582974 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6b2c2-a55a-4915-86eb-05a7b22e90cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:02.583705 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6b2c2-a55a-4915-86eb-05a7b22e90cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:02.585650 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6b2c2-a55a-4915-86eb-05a7b22e90cc admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:02.596808 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1247/2516] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:02.600877 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1a4ee-c2a6-46c9-98c0-1f98c2379729 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:02.611959 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1a4ee-c2a6-46c9-98c0-1f98c2379729 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:02.616577 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c22705-9501-4dc3-9e85-fb125e7f30cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:02.617875 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c22705-9501-4dc3-9e85-fb125e7f30cd admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:02.618165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c22705-9501-4dc3-9e85-fb125e7f30cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:02.618482 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c22705-9501-4dc3-9e85-fb125e7f30cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:02.619214 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c22705-9501-4dc3-9e85-fb125e7f30cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:02.620827 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c22705-9501-4dc3-9e85-fb125e7f30cd admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:02.631580 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1270/2517] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:02.635938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6b2c2-a55a-4915-86eb-05a7b22e90cc admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:02.647030 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6b2c2-a55a-4915-86eb-05a7b22e90cc admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:02.652255 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5138cd3-71a4-48a1-b30e-aabfc3159240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:02.653153 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5138cd3-71a4-48a1-b30e-aabfc3159240 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:02.653434 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5138cd3-71a4-48a1-b30e-aabfc3159240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:02.653969 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5138cd3-71a4-48a1-b30e-aabfc3159240 admin admin] PATH_INFO: `/v3/projects/97dc670f096448f3b084123d6b10fd42/users/00f4e58117f24d98ba94d1c64fda7f86/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:02.669813 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5138cd3-71a4-48a1-b30e-aabfc3159240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97dc670f096448f3b084123d6b10fd42, user_id=00f4e58117f24d98ba94d1c64fda7f86, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:02.672786 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5138cd3-71a4-48a1-b30e-aabfc3159240 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:02.686936 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1248/2518] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:12:02 2022] PUT /identity/v3/projects/97dc670f096448f3b084123d6b10fd42/users/00f4e58117f24d98ba94d1c64fda7f86/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:02.698795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488660a-0e4e-46be-83f6-5804ba8d2a2b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:02.699136 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488660a-0e4e-46be-83f6-5804ba8d2a2b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:02.699415 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488660a-0e4e-46be-83f6-5804ba8d2a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:02.725895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b488660a-0e4e-46be-83f6-5804ba8d2a2b None None] MFA Rules not processed for user `00f4e58117f24d98ba94d1c64fda7f86`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:02.728189 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b488660a-0e4e-46be-83f6-5804ba8d2a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:02.797290 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1271/2519] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:02 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:02.801723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5138cd3-71a4-48a1-b30e-aabfc3159240 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:02.864294 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5138cd3-71a4-48a1-b30e-aabfc3159240 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:02.870171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c166bdb-2157-4b26-b6cd-9262b85f160f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:02.870861 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c166bdb-2157-4b26-b6cd-9262b85f160f admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:02.871148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c166bdb-2157-4b26-b6cd-9262b85f160f admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:02.871428 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c166bdb-2157-4b26-b6cd-9262b85f160f admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:02.872143 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c166bdb-2157-4b26-b6cd-9262b85f160f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:02.874086 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c166bdb-2157-4b26-b6cd-9262b85f160f admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:02.899160 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1249/2520] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:12:02 2022] POST /identity/v3/projects => generated 409 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:02.903291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488660a-0e4e-46be-83f6-5804ba8d2a2b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:02.910262 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:02.915907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488660a-0e4e-46be-83f6-5804ba8d2a2b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:02.975419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546edc1-fff9-466e-9d0d-faa65f347dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:02.976093 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546edc1-fff9-466e-9d0d-faa65f347dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:02.977037 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546edc1-fff9-466e-9d0d-faa65f347dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:02.977474 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546edc1-fff9-466e-9d0d-faa65f347dc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:02.977944 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546edc1-fff9-466e-9d0d-faa65f347dc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:02.979921 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546edc1-fff9-466e-9d0d-faa65f347dc0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:02.999658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1272/2521] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:12:02 2022] POST /identity/v3/users => generated 311 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:12:03.003625 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c166bdb-2157-4b26-b6cd-9262b85f160f admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:03.013818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c166bdb-2157-4b26-b6cd-9262b85f160f admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:03.018444 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc54810-aa58-4119-b99e-7bbd7997917f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:03.019089 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc54810-aa58-4119-b99e-7bbd7997917f admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:03.019369 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc54810-aa58-4119-b99e-7bbd7997917f admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:03.019647 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc54810-aa58-4119-b99e-7bbd7997917f admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:03.020370 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc54810-aa58-4119-b99e-7bbd7997917f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:03.021954 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc54810-aa58-4119-b99e-7bbd7997917f admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:03.034247 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1250/2522] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:03.038377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546edc1-fff9-466e-9d0d-faa65f347dc0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:03.049398 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546edc1-fff9-466e-9d0d-faa65f347dc0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:03.054623 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ce443-7d22-460b-a277-e44b7935aebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:03.055508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ce443-7d22-460b-a277-e44b7935aebb admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:03.055787 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ce443-7d22-460b-a277-e44b7935aebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:03.056060 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ce443-7d22-460b-a277-e44b7935aebb admin admin] PATH_INFO: `/v3/projects/c87ba2b7bebf40ff91474eb0f2b01e81/users/715c24e4467f495883b374c297c58e33/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:03.070610 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18ce443-7d22-460b-a277-e44b7935aebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c87ba2b7bebf40ff91474eb0f2b01e81, user_id=715c24e4467f495883b374c297c58e33, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:03.073357 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18ce443-7d22-460b-a277-e44b7935aebb admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:03.087606 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1273/2523] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:12:03 2022] PUT /identity/v3/projects/c87ba2b7bebf40ff91474eb0f2b01e81/users/715c24e4467f495883b374c297c58e33/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:03.092036 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc54810-aa58-4119-b99e-7bbd7997917f admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:03.103560 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc54810-aa58-4119-b99e-7bbd7997917f admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:03.162108 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34878dc1-82e5-488c-a851-25fbc1ba14ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:03.162825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34878dc1-82e5-488c-a851-25fbc1ba14ae admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:03.163116 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34878dc1-82e5-488c-a851-25fbc1ba14ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:03.163425 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34878dc1-82e5-488c-a851-25fbc1ba14ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:03.164784 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34878dc1-82e5-488c-a851-25fbc1ba14ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:03.166608 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34878dc1-82e5-488c-a851-25fbc1ba14ae admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:03.176609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1251/2524] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:03.181355 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ce443-7d22-460b-a277-e44b7935aebb admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:03.191978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ce443-7d22-460b-a277-e44b7935aebb admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:03.197001 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debeec30-781b-43f9-9ed2-128ac00e8cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:03.197933 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeec30-781b-43f9-9ed2-128ac00e8cd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:03.198246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeec30-781b-43f9-9ed2-128ac00e8cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:03.198530 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeec30-781b-43f9-9ed2-128ac00e8cd5 admin admin] PATH_INFO: `/v3/projects/c87ba2b7bebf40ff91474eb0f2b01e81/users/715c24e4467f495883b374c297c58e33/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:03.200496 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debeec30-781b-43f9-9ed2-128ac00e8cd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c87ba2b7bebf40ff91474eb0f2b01e81, user_id=715c24e4467f495883b374c297c58e33, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:03.203181 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debeec30-781b-43f9-9ed2-128ac00e8cd5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:03.214711 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1274/2525] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:12:03 2022] PUT /identity/v3/projects/c87ba2b7bebf40ff91474eb0f2b01e81/users/715c24e4467f495883b374c297c58e33/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:12:03.224825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f7603-dc50-4d6d-93d1-05c24c9d6d46 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:03.225115 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f7603-dc50-4d6d-93d1-05c24c9d6d46 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:03.225426 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f7603-dc50-4d6d-93d1-05c24c9d6d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:03.253326 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-449f7603-dc50-4d6d-93d1-05c24c9d6d46 None None] MFA Rules not processed for user `715c24e4467f495883b374c297c58e33`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:03.254679 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-449f7603-dc50-4d6d-93d1-05c24c9d6d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:03.284948 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:03.327410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1252/2526] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:03 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:03.338699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debeec30-781b-43f9-9ed2-128ac00e8cd5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:03.340627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-debeec30-781b-43f9-9ed2-128ac00e8cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:03.402882 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debeec30-781b-43f9-9ed2-128ac00e8cd5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:03.406765 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480da79-f6a4-4c6e-a3b4-0d73e7d14df6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:03.407416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480da79-f6a4-4c6e-a3b4-0d73e7d14df6 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:03.407700 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480da79-f6a4-4c6e-a3b4-0d73e7d14df6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:03.407975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480da79-f6a4-4c6e-a3b4-0d73e7d14df6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:03.417916 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480da79-f6a4-4c6e-a3b4-0d73e7d14df6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:03.420091 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480da79-f6a4-4c6e-a3b4-0d73e7d14df6 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:03.432127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1275/2527] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:03 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:03.585160 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f7603-dc50-4d6d-93d1-05c24c9d6d46 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:03.587435 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-449f7603-dc50-4d6d-93d1-05c24c9d6d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:03.599845 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f7603-dc50-4d6d-93d1-05c24c9d6d46 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:03.604649 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a69ac-cdef-4193-adb2-e4a185258abf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:03.605352 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a69ac-cdef-4193-adb2-e4a185258abf service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:03.605641 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a69ac-cdef-4193-adb2-e4a185258abf service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:03.605924 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a69ac-cdef-4193-adb2-e4a185258abf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:03.664548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a69ac-cdef-4193-adb2-e4a185258abf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:03.666003 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a69ac-cdef-4193-adb2-e4a185258abf service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:03.675133 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1253/2528] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:03 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:04.514686 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480da79-f6a4-4c6e-a3b4-0d73e7d14df6 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:04.516144 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2480da79-f6a4-4c6e-a3b4-0d73e7d14df6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:04.583419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480da79-f6a4-4c6e-a3b4-0d73e7d14df6 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:04.589107 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3f1c8-7beb-4959-ac72-f63ff1937a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:04.589972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3f1c8-7beb-4959-ac72-f63ff1937a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:04.590303 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3f1c8-7beb-4959-ac72-f63ff1937a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:04.590561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3f1c8-7beb-4959-ac72-f63ff1937a3e admin admin] PATH_INFO: `/v3/users/c03d2a22086a41138bf038ebab42e0fd` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:04.591734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3f1c8-7beb-4959-ac72-f63ff1937a3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c03d2a22086a41138bf038ebab42e0fd)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:04.593653 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3f1c8-7beb-4959-ac72-f63ff1937a3e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:04.595189 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:04.641204 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-abb3f1c8-7beb-4959-ac72-f63ff1937a3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c03d2a22086a41138bf038ebab42e0fd'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:04.649006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-abb3f1c8-7beb-4959-ac72-f63ff1937a3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c03d2a22086a41138bf038ebab42e0fd'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:04.661231 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-abb3f1c8-7beb-4959-ac72-f63ff1937a3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c03d2a22086a41138bf038ebab42e0fd'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:04.661874 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-abb3f1c8-7beb-4959-ac72-f63ff1937a3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c03d2a22086a41138bf038ebab42e0fd'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:04.662493 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-abb3f1c8-7beb-4959-ac72-f63ff1937a3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c03d2a22086a41138bf038ebab42e0fd'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:04.675555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1276/2529] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:12:04 2022] DELETE /identity/v3/users/c03d2a22086a41138bf038ebab42e0fd => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:04.680248 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc942fe6-2ef5-41b9-a425-eb7d25c450c2 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:04.680835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc942fe6-2ef5-41b9-a425-eb7d25c450c2 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:04.680835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc942fe6-2ef5-41b9-a425-eb7d25c450c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:04.697579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-dc942fe6-2ef5-41b9-a425-eb7d25c450c2 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:04.698851 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-dc942fe6-2ef5-41b9-a425-eb7d25c450c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:04.756531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1254/2530] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:04.766277 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3f1c8-7beb-4959-ac72-f63ff1937a3e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:04.767684 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-abb3f1c8-7beb-4959-ac72-f63ff1937a3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:04.829784 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3f1c8-7beb-4959-ac72-f63ff1937a3e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:04.833427 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bf7c9-37a6-477e-8d03-781d40103f8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:04.834084 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bf7c9-37a6-477e-8d03-781d40103f8d service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:04.834394 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bf7c9-37a6-477e-8d03-781d40103f8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:04.834668 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bf7c9-37a6-477e-8d03-781d40103f8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:04.844837 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9bf7c9-37a6-477e-8d03-781d40103f8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:04.846293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9bf7c9-37a6-477e-8d03-781d40103f8d service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:04.855857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1277/2531] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:05.145893 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc942fe6-2ef5-41b9-a425-eb7d25c450c2 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:05.147857 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-dc942fe6-2ef5-41b9-a425-eb7d25c450c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:05.160824 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc942fe6-2ef5-41b9-a425-eb7d25c450c2 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:05.166335 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515c5f4-f7ac-4577-b256-3ca4fb19b0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:05.167386 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515c5f4-f7ac-4577-b256-3ca4fb19b0f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:05.167787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515c5f4-f7ac-4577-b256-3ca4fb19b0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:05.168117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515c5f4-f7ac-4577-b256-3ca4fb19b0f8 admin admin] PATH_INFO: `/v3/projects/ea248afbc39f4c4ba9dbcbfb03b46731` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:05.169573 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2515c5f4-f7ac-4577-b256-3ca4fb19b0f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea248afbc39f4c4ba9dbcbfb03b46731)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:05.171714 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2515c5f4-f7ac-4577-b256-3ca4fb19b0f8 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:05.235248 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2515c5f4-f7ac-4577-b256-3ca4fb19b0f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea248afbc39f4c4ba9dbcbfb03b46731', 'request_id': 'req-2515c5f4-f7ac-4577-b256-3ca4fb19b0f8'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:05.244406 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2515c5f4-f7ac-4577-b256-3ca4fb19b0f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea248afbc39f4c4ba9dbcbfb03b46731', 'request_id': 'req-2515c5f4-f7ac-4577-b256-3ca4fb19b0f8'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:05.246687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:05.251393 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2515c5f4-f7ac-4577-b256-3ca4fb19b0f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea248afbc39f4c4ba9dbcbfb03b46731', 'request_id': 'req-2515c5f4-f7ac-4577-b256-3ca4fb19b0f8'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:05.265386 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2515c5f4-f7ac-4577-b256-3ca4fb19b0f8 admin admin] The token cache is being invalidate because project ea248afbc39f4c4ba9dbcbfb03b46731 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:05.265603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2515c5f4-f7ac-4577-b256-3ca4fb19b0f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:05.266281 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2515c5f4-f7ac-4577-b256-3ca4fb19b0f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:05.268144 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1255/2532] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:12:05 2022] DELETE /identity/v3/projects/ea248afbc39f4c4ba9dbcbfb03b46731 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:05.273466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782131b-40d6-45c7-9403-01aff9a76337 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:05.273764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782131b-40d6-45c7-9403-01aff9a76337 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:05.274045 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782131b-40d6-45c7-9403-01aff9a76337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:05.292360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b782131b-40d6-45c7-9403-01aff9a76337 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:05.293562 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b782131b-40d6-45c7-9403-01aff9a76337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:05.352010 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1278/2533] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:05.359601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb21d9-10a8-4934-b833-f645e65829ac None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:05.359919 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb21d9-10a8-4934-b833-f645e65829ac None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:05.360201 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb21d9-10a8-4934-b833-f645e65829ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:05.379422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-eddb21d9-10a8-4934-b833-f645e65829ac None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:05.380621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-eddb21d9-10a8-4934-b833-f645e65829ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:05.389664 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1256/2534] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:05.393874 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782131b-40d6-45c7-9403-01aff9a76337 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:05.403975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782131b-40d6-45c7-9403-01aff9a76337 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:05.408547 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9d15c-8cc2-4350-9ba5-bcc226b46794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:05.409201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9d15c-8cc2-4350-9ba5-bcc226b46794 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:05.409487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9d15c-8cc2-4350-9ba5-bcc226b46794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:05.409762 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9d15c-8cc2-4350-9ba5-bcc226b46794 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:05.410522 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9d15c-8cc2-4350-9ba5-bcc226b46794 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:12:05.410828 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9d15c-8cc2-4350-9ba5-bcc226b46794 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:05.412482 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9d15c-8cc2-4350-9ba5-bcc226b46794 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:05.423980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1279/2535] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:12:05.428223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb21d9-10a8-4934-b833-f645e65829ac None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:05.438397 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb21d9-10a8-4934-b833-f645e65829ac None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:05.442996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657eec9e-eecc-47ef-b8ee-9cea2a861931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:05.443641 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657eec9e-eecc-47ef-b8ee-9cea2a861931 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:05.443924 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657eec9e-eecc-47ef-b8ee-9cea2a861931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:05.444199 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657eec9e-eecc-47ef-b8ee-9cea2a861931 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:05.444895 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657eec9e-eecc-47ef-b8ee-9cea2a861931 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:05.448042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657eec9e-eecc-47ef-b8ee-9cea2a861931 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:05.473259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1257/2536] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:12:05 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:05.477396 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9d15c-8cc2-4350-9ba5-bcc226b46794 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:05.487460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9d15c-8cc2-4350-9ba5-bcc226b46794 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:05.493068 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:05.547463 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f86f0-37bc-4553-a558-1a17b09754ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:05.548092 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f86f0-37bc-4553-a558-1a17b09754ef admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:05.548310 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f86f0-37bc-4553-a558-1a17b09754ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:05.548605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f86f0-37bc-4553-a558-1a17b09754ef admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:05.549295 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f86f0-37bc-4553-a558-1a17b09754ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:05.551246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f86f0-37bc-4553-a558-1a17b09754ef admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:05.576615 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1280/2537] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:12:05 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:12:05.577816 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657eec9e-eecc-47ef-b8ee-9cea2a861931 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:05.588120 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657eec9e-eecc-47ef-b8ee-9cea2a861931 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:05.592749 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e6c3d-4b73-406b-b9a3-d68ef4a1942a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:05.593424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e6c3d-4b73-406b-b9a3-d68ef4a1942a admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:05.593692 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e6c3d-4b73-406b-b9a3-d68ef4a1942a admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:05.593970 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e6c3d-4b73-406b-b9a3-d68ef4a1942a admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:05.594732 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e6c3d-4b73-406b-b9a3-d68ef4a1942a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:05.596332 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e6c3d-4b73-406b-b9a3-d68ef4a1942a admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:05.607613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1258/2538] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:05.611628 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f86f0-37bc-4553-a558-1a17b09754ef admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:05.622480 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f86f0-37bc-4553-a558-1a17b09754ef admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:05.627087 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed6bca-e64d-4a67-af5a-bf7ec993b84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:05.627743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed6bca-e64d-4a67-af5a-bf7ec993b84f admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:05.628051 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed6bca-e64d-4a67-af5a-bf7ec993b84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:05.628310 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed6bca-e64d-4a67-af5a-bf7ec993b84f admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:05.629027 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed6bca-e64d-4a67-af5a-bf7ec993b84f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:05.630689 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed6bca-e64d-4a67-af5a-bf7ec993b84f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:05.640779 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1281/2539] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:05.645051 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e6c3d-4b73-406b-b9a3-d68ef4a1942a admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:05.655234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e6c3d-4b73-406b-b9a3-d68ef4a1942a admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:05.660724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6194b-decc-4b65-adaa-364c24fcb101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:05.661618 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6194b-decc-4b65-adaa-364c24fcb101 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:05.661909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6194b-decc-4b65-adaa-364c24fcb101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:05.662208 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6194b-decc-4b65-adaa-364c24fcb101 admin admin] PATH_INFO: `/v3/projects/ead15c2be6014ba2907258fd5d1c1355/users/4f2a7ff22b094a43ad08992ac6cb03d6/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:05.677532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6194b-decc-4b65-adaa-364c24fcb101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ead15c2be6014ba2907258fd5d1c1355, user_id=4f2a7ff22b094a43ad08992ac6cb03d6, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:05.680850 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6194b-decc-4b65-adaa-364c24fcb101 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:05.694507 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1259/2540] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:12:05 2022] PUT /identity/v3/projects/ead15c2be6014ba2907258fd5d1c1355/users/4f2a7ff22b094a43ad08992ac6cb03d6/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:05.704849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991adbb-521b-47cd-9ba3-1c211b372aca None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:05.705150 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991adbb-521b-47cd-9ba3-1c211b372aca None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:05.705437 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991adbb-521b-47cd-9ba3-1c211b372aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:05.732625 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4991adbb-521b-47cd-9ba3-1c211b372aca None None] MFA Rules not processed for user `4f2a7ff22b094a43ad08992ac6cb03d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:05.733861 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4991adbb-521b-47cd-9ba3-1c211b372aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:05.799386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1282/2541] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:05.803878 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6194b-decc-4b65-adaa-364c24fcb101 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:05.865693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6194b-decc-4b65-adaa-364c24fcb101 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:05.870638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c493f-6cf8-48e2-b05a-36ad10248752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:05.871406 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c493f-6cf8-48e2-b05a-36ad10248752 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:05.871695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c493f-6cf8-48e2-b05a-36ad10248752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:05.872029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c493f-6cf8-48e2-b05a-36ad10248752 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:05.872746 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c493f-6cf8-48e2-b05a-36ad10248752 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:05.874675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c493f-6cf8-48e2-b05a-36ad10248752 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:05.900935 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1260/2542] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:12:05 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:12:05.905720 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991adbb-521b-47cd-9ba3-1c211b372aca None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:05.916508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991adbb-521b-47cd-9ba3-1c211b372aca None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:05.976626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e4477-f648-4eab-9f1a-b5e0817440cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:05.977394 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e4477-f648-4eab-9f1a-b5e0817440cb admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:05.977663 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e4477-f648-4eab-9f1a-b5e0817440cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:05.977965 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e4477-f648-4eab-9f1a-b5e0817440cb admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:05.978764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e4477-f648-4eab-9f1a-b5e0817440cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:05.980633 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e4477-f648-4eab-9f1a-b5e0817440cb admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:06.003152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1283/2543] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:12:05 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:06.007198 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c493f-6cf8-48e2-b05a-36ad10248752 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:06.019664 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c493f-6cf8-48e2-b05a-36ad10248752 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:06.026088 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52bd354-bf29-4535-9e70-c5400666e97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:06.026824 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bd354-bf29-4535-9e70-c5400666e97e admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:06.027116 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bd354-bf29-4535-9e70-c5400666e97e admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:06.027397 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bd354-bf29-4535-9e70-c5400666e97e admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:06.028133 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52bd354-bf29-4535-9e70-c5400666e97e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:06.030002 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52bd354-bf29-4535-9e70-c5400666e97e admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:06.041090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1261/2544] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:06.045412 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e4477-f648-4eab-9f1a-b5e0817440cb admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:06.056938 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e4477-f648-4eab-9f1a-b5e0817440cb admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:06.061901 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36db7b-8dbd-449d-8fb2-00d2998be890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:06.063008 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36db7b-8dbd-449d-8fb2-00d2998be890 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:06.063382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36db7b-8dbd-449d-8fb2-00d2998be890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:06.063584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36db7b-8dbd-449d-8fb2-00d2998be890 admin admin] PATH_INFO: `/v3/projects/1e245788fb274afeaf1ae201777ae4bf/users/aba7789d3f934c3b898b3c4cbe2b4a85/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:06.067152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:06.083376 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36db7b-8dbd-449d-8fb2-00d2998be890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e245788fb274afeaf1ae201777ae4bf, user_id=aba7789d3f934c3b898b3c4cbe2b4a85, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:06.086474 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36db7b-8dbd-449d-8fb2-00d2998be890 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:06.100132 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1284/2545] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:12:06 2022] PUT /identity/v3/projects/1e245788fb274afeaf1ae201777ae4bf/users/aba7789d3f934c3b898b3c4cbe2b4a85/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:06.105358 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52bd354-bf29-4535-9e70-c5400666e97e admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:06.115590 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52bd354-bf29-4535-9e70-c5400666e97e admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:06.173111 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:06.181721 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24530e4f-602a-47ae-bd00-0c6de77a1347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:06.182847 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24530e4f-602a-47ae-bd00-0c6de77a1347 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:06.183208 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24530e4f-602a-47ae-bd00-0c6de77a1347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:06.183585 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24530e4f-602a-47ae-bd00-0c6de77a1347 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:06.184591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24530e4f-602a-47ae-bd00-0c6de77a1347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:06.186719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24530e4f-602a-47ae-bd00-0c6de77a1347 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:06.198298 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1262/2546] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:06.202689 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36db7b-8dbd-449d-8fb2-00d2998be890 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:06.213845 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36db7b-8dbd-449d-8fb2-00d2998be890 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:06.219277 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5c580-6536-41e6-983d-c25fa018ee65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:06.220905 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5c580-6536-41e6-983d-c25fa018ee65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:06.221204 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5c580-6536-41e6-983d-c25fa018ee65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:06.221489 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5c580-6536-41e6-983d-c25fa018ee65 admin admin] PATH_INFO: `/v3/projects/1e245788fb274afeaf1ae201777ae4bf/users/aba7789d3f934c3b898b3c4cbe2b4a85/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:06.223854 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5c580-6536-41e6-983d-c25fa018ee65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e245788fb274afeaf1ae201777ae4bf, user_id=aba7789d3f934c3b898b3c4cbe2b4a85, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:06.227473 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5c580-6536-41e6-983d-c25fa018ee65 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:06.241629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1285/2547] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:12:06 2022] PUT /identity/v3/projects/1e245788fb274afeaf1ae201777ae4bf/users/aba7789d3f934c3b898b3c4cbe2b4a85/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:06.254548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152e015-a115-4f8a-99af-4706da1ecd51 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:06.254847 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152e015-a115-4f8a-99af-4706da1ecd51 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:06.255149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152e015-a115-4f8a-99af-4706da1ecd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:06.286897 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-f152e015-a115-4f8a-99af-4706da1ecd51 None None] MFA Rules not processed for user `aba7789d3f934c3b898b3c4cbe2b4a85`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:06.288137 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f152e015-a115-4f8a-99af-4706da1ecd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:06.374936 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1263/2548] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:06 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:06.386724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5c580-6536-41e6-983d-c25fa018ee65 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:06.388431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-76e5c580-6536-41e6-983d-c25fa018ee65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:06.447537 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5c580-6536-41e6-983d-c25fa018ee65 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:06.451626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13c88e-9391-4ab9-a36d-43fb11c12ddc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:06.452276 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13c88e-9391-4ab9-a36d-43fb11c12ddc service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:06.452566 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13c88e-9391-4ab9-a36d-43fb11c12ddc service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:06.452848 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13c88e-9391-4ab9-a36d-43fb11c12ddc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:06.462205 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e13c88e-9391-4ab9-a36d-43fb11c12ddc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:06.463650 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e13c88e-9391-4ab9-a36d-43fb11c12ddc service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:06.473253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1286/2549] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:06 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:06.546480 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152e015-a115-4f8a-99af-4706da1ecd51 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:06.556783 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152e015-a115-4f8a-99af-4706da1ecd51 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:06.560643 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088dcf2-a1f7-480c-9fae-db60b50c2d11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:06.561296 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088dcf2-a1f7-480c-9fae-db60b50c2d11 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:06.561585 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088dcf2-a1f7-480c-9fae-db60b50c2d11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:06.561864 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088dcf2-a1f7-480c-9fae-db60b50c2d11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:06.619756 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2088dcf2-a1f7-480c-9fae-db60b50c2d11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:06.621112 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2088dcf2-a1f7-480c-9fae-db60b50c2d11 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:06.630149 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1264/2550] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:06 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:10.364258 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13c88e-9391-4ab9-a36d-43fb11c12ddc service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:10.428913 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13c88e-9391-4ab9-a36d-43fb11c12ddc service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:10.433957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea931bb-8a7d-4449-93af-4fd1186e1461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:10.434841 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea931bb-8a7d-4449-93af-4fd1186e1461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:10.435126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea931bb-8a7d-4449-93af-4fd1186e1461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:10.435403 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea931bb-8a7d-4449-93af-4fd1186e1461 admin admin] PATH_INFO: `/v3/users/4f2a7ff22b094a43ad08992ac6cb03d6` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:10.436597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea931bb-8a7d-4449-93af-4fd1186e1461 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f2a7ff22b094a43ad08992ac6cb03d6)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:10.438559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea931bb-8a7d-4449-93af-4fd1186e1461 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:10.485991 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cea931bb-8a7d-4449-93af-4fd1186e1461 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f2a7ff22b094a43ad08992ac6cb03d6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:10.493250 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cea931bb-8a7d-4449-93af-4fd1186e1461 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f2a7ff22b094a43ad08992ac6cb03d6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:10.504872 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cea931bb-8a7d-4449-93af-4fd1186e1461 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f2a7ff22b094a43ad08992ac6cb03d6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:10.505516 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cea931bb-8a7d-4449-93af-4fd1186e1461 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f2a7ff22b094a43ad08992ac6cb03d6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:10.506110 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-cea931bb-8a7d-4449-93af-4fd1186e1461 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f2a7ff22b094a43ad08992ac6cb03d6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:10.508444 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:10.520598 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1287/2551] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:12:10 2022] DELETE /identity/v3/users/4f2a7ff22b094a43ad08992ac6cb03d6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:10.525962 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092cd13-eb56-4849-b16f-fd804e829422 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:10.526507 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092cd13-eb56-4849-b16f-fd804e829422 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:10.526625 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092cd13-eb56-4849-b16f-fd804e829422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:10.544473 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-d092cd13-eb56-4849-b16f-fd804e829422 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:10.545679 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-d092cd13-eb56-4849-b16f-fd804e829422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:10.610539 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1265/2552] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:10.620384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea931bb-8a7d-4449-93af-4fd1186e1461 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:10.621813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cea931bb-8a7d-4449-93af-4fd1186e1461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:10.682816 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea931bb-8a7d-4449-93af-4fd1186e1461 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:10.686486 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55a51e-5c75-409e-a71f-2615999e3498 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:10.687142 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55a51e-5c75-409e-a71f-2615999e3498 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:10.687430 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55a51e-5c75-409e-a71f-2615999e3498 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:10.687710 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55a51e-5c75-409e-a71f-2615999e3498 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:10.698669 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d55a51e-5c75-409e-a71f-2615999e3498 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:10.700109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d55a51e-5c75-409e-a71f-2615999e3498 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:10.709784 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1288/2553] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:10.999003 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092cd13-eb56-4849-b16f-fd804e829422 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:11.000509 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-d092cd13-eb56-4849-b16f-fd804e829422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:11.014151 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092cd13-eb56-4849-b16f-fd804e829422 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:11.020065 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff55057-8599-4cbd-abf8-4189ef4578e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:11.020954 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff55057-8599-4cbd-abf8-4189ef4578e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:11.021277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff55057-8599-4cbd-abf8-4189ef4578e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:11.021557 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff55057-8599-4cbd-abf8-4189ef4578e4 admin admin] PATH_INFO: `/v3/projects/ead15c2be6014ba2907258fd5d1c1355` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:11.022840 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff55057-8599-4cbd-abf8-4189ef4578e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ead15c2be6014ba2907258fd5d1c1355)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:11.025755 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff55057-8599-4cbd-abf8-4189ef4578e4 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:11.038930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:11.079162 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-9ff55057-8599-4cbd-abf8-4189ef4578e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ead15c2be6014ba2907258fd5d1c1355', 'request_id': 'req-9ff55057-8599-4cbd-abf8-4189ef4578e4'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:11.088498 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-9ff55057-8599-4cbd-abf8-4189ef4578e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ead15c2be6014ba2907258fd5d1c1355', 'request_id': 'req-9ff55057-8599-4cbd-abf8-4189ef4578e4'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:11.094825 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-9ff55057-8599-4cbd-abf8-4189ef4578e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ead15c2be6014ba2907258fd5d1c1355', 'request_id': 'req-9ff55057-8599-4cbd-abf8-4189ef4578e4'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:11.107318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-9ff55057-8599-4cbd-abf8-4189ef4578e4 admin admin] The token cache is being invalidate because project ead15c2be6014ba2907258fd5d1c1355 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:11.107607 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-9ff55057-8599-4cbd-abf8-4189ef4578e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:11.108231 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-9ff55057-8599-4cbd-abf8-4189ef4578e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:11.110367 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1266/2554] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:12:10 2022] DELETE /identity/v3/projects/ead15c2be6014ba2907258fd5d1c1355 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:11.114562 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55a51e-5c75-409e-a71f-2615999e3498 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:11.124595 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55a51e-5c75-409e-a71f-2615999e3498 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:11.188183 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5ca74-ea72-4eba-98f8-cd6600ab9946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:11.189050 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5ca74-ea72-4eba-98f8-cd6600ab9946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:11.189339 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5ca74-ea72-4eba-98f8-cd6600ab9946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:11.189615 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5ca74-ea72-4eba-98f8-cd6600ab9946 admin admin] PATH_INFO: `/v3/users/aba7789d3f934c3b898b3c4cbe2b4a85` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:11.190821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5ca74-ea72-4eba-98f8-cd6600ab9946 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aba7789d3f934c3b898b3c4cbe2b4a85)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:11.192828 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5ca74-ea72-4eba-98f8-cd6600ab9946 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:11.239772 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-29c5ca74-ea72-4eba-98f8-cd6600ab9946 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aba7789d3f934c3b898b3c4cbe2b4a85'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:11.247923 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-29c5ca74-ea72-4eba-98f8-cd6600ab9946 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aba7789d3f934c3b898b3c4cbe2b4a85'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:11.260463 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-29c5ca74-ea72-4eba-98f8-cd6600ab9946 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aba7789d3f934c3b898b3c4cbe2b4a85'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:11.261105 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-29c5ca74-ea72-4eba-98f8-cd6600ab9946 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aba7789d3f934c3b898b3c4cbe2b4a85'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:11.261707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-29c5ca74-ea72-4eba-98f8-cd6600ab9946 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aba7789d3f934c3b898b3c4cbe2b4a85'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:11.273743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1289/2555] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:12:11 2022] DELETE /identity/v3/users/aba7789d3f934c3b898b3c4cbe2b4a85 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:11.551479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff55057-8599-4cbd-abf8-4189ef4578e4 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:11.552958 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-9ff55057-8599-4cbd-abf8-4189ef4578e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:11.568199 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff55057-8599-4cbd-abf8-4189ef4578e4 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:11.627916 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985ebe4-ac35-4ffe-9c88-271e4b272cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:11.628810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985ebe4-ac35-4ffe-9c88-271e4b272cc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:11.629089 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985ebe4-ac35-4ffe-9c88-271e4b272cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:11.629359 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985ebe4-ac35-4ffe-9c88-271e4b272cc7 admin admin] PATH_INFO: `/v3/projects/1e245788fb274afeaf1ae201777ae4bf` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:11.630565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3985ebe4-ac35-4ffe-9c88-271e4b272cc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e245788fb274afeaf1ae201777ae4bf)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:11.633964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3985ebe4-ac35-4ffe-9c88-271e4b272cc7 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:11.679725 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-3985ebe4-ac35-4ffe-9c88-271e4b272cc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e245788fb274afeaf1ae201777ae4bf', 'request_id': 'req-3985ebe4-ac35-4ffe-9c88-271e4b272cc7'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:11.687787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-3985ebe4-ac35-4ffe-9c88-271e4b272cc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e245788fb274afeaf1ae201777ae4bf', 'request_id': 'req-3985ebe4-ac35-4ffe-9c88-271e4b272cc7'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:11.693894 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-3985ebe4-ac35-4ffe-9c88-271e4b272cc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e245788fb274afeaf1ae201777ae4bf', 'request_id': 'req-3985ebe4-ac35-4ffe-9c88-271e4b272cc7'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:11.705390 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-3985ebe4-ac35-4ffe-9c88-271e4b272cc7 admin admin] The token cache is being invalidate because project 1e245788fb274afeaf1ae201777ae4bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:11.705675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-3985ebe4-ac35-4ffe-9c88-271e4b272cc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:11.706347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-3985ebe4-ac35-4ffe-9c88-271e4b272cc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:11.707892 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1267/2556] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:12:11 2022] DELETE /identity/v3/projects/1e245788fb274afeaf1ae201777ae4bf => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:11.713135 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfae8a1-54ed-41ea-81fe-2cf4f7fc49aa None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:11.713616 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfae8a1-54ed-41ea-81fe-2cf4f7fc49aa None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:11.713816 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfae8a1-54ed-41ea-81fe-2cf4f7fc49aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:11.734095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3bfae8a1-54ed-41ea-81fe-2cf4f7fc49aa None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:11.735354 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3bfae8a1-54ed-41ea-81fe-2cf4f7fc49aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:11.746365 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:11.805668 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1290/2557] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:12:11.812852 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0462e9c-df94-434d-bc12-36d20e480480 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:11.813217 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0462e9c-df94-434d-bc12-36d20e480480 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:11.813607 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0462e9c-df94-434d-bc12-36d20e480480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:11.832834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-e0462e9c-df94-434d-bc12-36d20e480480 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:11.834081 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-e0462e9c-df94-434d-bc12-36d20e480480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:11.842747 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1268/2558] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:11.846948 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfae8a1-54ed-41ea-81fe-2cf4f7fc49aa None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:11.857893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfae8a1-54ed-41ea-81fe-2cf4f7fc49aa None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:11.862589 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d2168-332d-4dfd-aeb2-e0da0635e654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:11.863254 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d2168-332d-4dfd-aeb2-e0da0635e654 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:11.863537 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d2168-332d-4dfd-aeb2-e0da0635e654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:11.863956 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d2168-332d-4dfd-aeb2-e0da0635e654 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:11.864800 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d2168-332d-4dfd-aeb2-e0da0635e654 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:12:11.865278 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d2168-332d-4dfd-aeb2-e0da0635e654 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:11.868482 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d2168-332d-4dfd-aeb2-e0da0635e654 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:11.882286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1291/2559] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:12:11.887128 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0462e9c-df94-434d-bc12-36d20e480480 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:11.897113 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0462e9c-df94-434d-bc12-36d20e480480 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:11.908350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c9a36-81c7-4aef-8cb0-3577f2f3c5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:11.909222 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c9a36-81c7-4aef-8cb0-3577f2f3c5fd admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:11.918872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c9a36-81c7-4aef-8cb0-3577f2f3c5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:11.919154 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c9a36-81c7-4aef-8cb0-3577f2f3c5fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:11.920151 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c9a36-81c7-4aef-8cb0-3577f2f3c5fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:11.922820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c9a36-81c7-4aef-8cb0-3577f2f3c5fd admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:11.951587 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1269/2560] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:12:11 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:11.955560 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d2168-332d-4dfd-aeb2-e0da0635e654 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:11.967152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d2168-332d-4dfd-aeb2-e0da0635e654 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:12.029657 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e255300-1e9f-4606-8a50-7e4152b1bedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:12.030362 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e255300-1e9f-4606-8a50-7e4152b1bedd admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.030647 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e255300-1e9f-4606-8a50-7e4152b1bedd admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.030918 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e255300-1e9f-4606-8a50-7e4152b1bedd admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.031618 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e255300-1e9f-4606-8a50-7e4152b1bedd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:12.033671 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e255300-1e9f-4606-8a50-7e4152b1bedd admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:12.057457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1292/2561] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:12:11 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:12.058703 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c9a36-81c7-4aef-8cb0-3577f2f3c5fd admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:12.064277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:12.072484 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c9a36-81c7-4aef-8cb0-3577f2f3c5fd admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:12.073347 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e255300-1e9f-4606-8a50-7e4152b1bedd admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:12.075368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6e255300-1e9f-4606-8a50-7e4152b1bedd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:12.077823 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe698528-dde2-45fd-a31a-101d4b3ecfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:12.078616 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe698528-dde2-45fd-a31a-101d4b3ecfae admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.078973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe698528-dde2-45fd-a31a-101d4b3ecfae admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.079327 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe698528-dde2-45fd-a31a-101d4b3ecfae admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.080157 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe698528-dde2-45fd-a31a-101d4b3ecfae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:12.081861 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe698528-dde2-45fd-a31a-101d4b3ecfae admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:12.087905 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e255300-1e9f-4606-8a50-7e4152b1bedd admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:12.092584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f65d4-b8c7-4b80-9acc-89a1d7f79da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:12.093469 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f65d4-b8c7-4b80-9acc-89a1d7f79da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.093759 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f65d4-b8c7-4b80-9acc-89a1d7f79da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.094041 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f65d4-b8c7-4b80-9acc-89a1d7f79da2 admin admin] PATH_INFO: `/v3/users/00f4e58117f24d98ba94d1c64fda7f86` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.097901 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1270/2562] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:12.098060 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f65d4-b8c7-4b80-9acc-89a1d7f79da2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00f4e58117f24d98ba94d1c64fda7f86)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:12.098060 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f65d4-b8c7-4b80-9acc-89a1d7f79da2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:12.099443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe698528-dde2-45fd-a31a-101d4b3ecfae admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:12.110246 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe698528-dde2-45fd-a31a-101d4b3ecfae admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:12.115044 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eda012-544f-446f-92ee-864568bb8b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:12.115727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eda012-544f-446f-92ee-864568bb8b3a admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.116021 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eda012-544f-446f-92ee-864568bb8b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.116292 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eda012-544f-446f-92ee-864568bb8b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.117230 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eda012-544f-446f-92ee-864568bb8b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:12.118909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eda012-544f-446f-92ee-864568bb8b3a admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:12.130558 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1271/2563] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:12:12.135135 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eda012-544f-446f-92ee-864568bb8b3a admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:12.146158 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eda012-544f-446f-92ee-864568bb8b3a admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:12.151641 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592eea4b-49bd-4d4a-9a3b-25325e4e65ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:12.152542 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592eea4b-49bd-4d4a-9a3b-25325e4e65ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.152827 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592eea4b-49bd-4d4a-9a3b-25325e4e65ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.153108 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592eea4b-49bd-4d4a-9a3b-25325e4e65ac admin admin] PATH_INFO: `/v3/projects/751c5807fa834ced9a9dd32ff99da2f6/users/bc75518bc7634a41b2dec8fd30b22ac8/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.153997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ad6f65d4-b8c7-4b80-9acc-89a1d7f79da2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00f4e58117f24d98ba94d1c64fda7f86'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:12.161903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ad6f65d4-b8c7-4b80-9acc-89a1d7f79da2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00f4e58117f24d98ba94d1c64fda7f86'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:12.169729 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592eea4b-49bd-4d4a-9a3b-25325e4e65ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=751c5807fa834ced9a9dd32ff99da2f6, user_id=bc75518bc7634a41b2dec8fd30b22ac8, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:12.172430 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592eea4b-49bd-4d4a-9a3b-25325e4e65ac admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:12.175095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ad6f65d4-b8c7-4b80-9acc-89a1d7f79da2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00f4e58117f24d98ba94d1c64fda7f86'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:12.175764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ad6f65d4-b8c7-4b80-9acc-89a1d7f79da2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00f4e58117f24d98ba94d1c64fda7f86'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:12.176405 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ad6f65d4-b8c7-4b80-9acc-89a1d7f79da2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00f4e58117f24d98ba94d1c64fda7f86'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:12.187644 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1272/2564] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:12:12 2022] PUT /identity/v3/projects/751c5807fa834ced9a9dd32ff99da2f6/users/bc75518bc7634a41b2dec8fd30b22ac8/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:12:12.190537 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1293/2565] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:12:12 2022] DELETE /identity/v3/users/00f4e58117f24d98ba94d1c64fda7f86 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:12.195442 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8ee52-93dc-44f2-95c0-e84f9c68c6d6 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.195733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8ee52-93dc-44f2-95c0-e84f9c68c6d6 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.196029 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8ee52-93dc-44f2-95c0-e84f9c68c6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.199840 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e58803-7092-435c-aa1d-30e12f22c90d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.200323 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e58803-7092-435c-aa1d-30e12f22c90d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.200779 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e58803-7092-435c-aa1d-30e12f22c90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.207973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:12.215844 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-07e8ee52-93dc-44f2-95c0-e84f9c68c6d6 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:12.217082 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-07e8ee52-93dc-44f2-95c0-e84f9c68c6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:12.233325 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-63e58803-7092-435c-aa1d-30e12f22c90d None None] MFA Rules not processed for user `bc75518bc7634a41b2dec8fd30b22ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:12.234567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-63e58803-7092-435c-aa1d-30e12f22c90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:12.280464 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1273/2566] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:12.290884 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8ee52-93dc-44f2-95c0-e84f9c68c6d6 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:12.292328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-07e8ee52-93dc-44f2-95c0-e84f9c68c6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:12.301411 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1294/2567] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:12.306361 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e58803-7092-435c-aa1d-30e12f22c90d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:12.307674 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-63e58803-7092-435c-aa1d-30e12f22c90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:12.321443 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e58803-7092-435c-aa1d-30e12f22c90d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:12.326206 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7ba46-3876-4ee5-bcac-450fb7f76bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:12.326872 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7ba46-3876-4ee5-bcac-450fb7f76bb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.327158 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7ba46-3876-4ee5-bcac-450fb7f76bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.327439 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7ba46-3876-4ee5-bcac-450fb7f76bb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.328138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7ba46-3876-4ee5-bcac-450fb7f76bb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:12.330072 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7ba46-3876-4ee5-bcac-450fb7f76bb1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:12.352016 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8ee52-93dc-44f2-95c0-e84f9c68c6d6 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:12.355672 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df67a8-b4f2-401c-91ca-6585c252397b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:12.356325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df67a8-b4f2-401c-91ca-6585c252397b service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.356615 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df67a8-b4f2-401c-91ca-6585c252397b service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.356895 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df67a8-b4f2-401c-91ca-6585c252397b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.360334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1295/2568] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:12:12 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:12:12.361300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7ba46-3876-4ee5-bcac-450fb7f76bb1 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:12.366587 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df67a8-b4f2-401c-91ca-6585c252397b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:12.368767 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df67a8-b4f2-401c-91ca-6585c252397b service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:12.379061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1274/2569] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:12:12.430535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7ba46-3876-4ee5-bcac-450fb7f76bb1 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:12.435440 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74eb6cb-8ae2-4e84-8b34-c73af78817a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:12.436115 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74eb6cb-8ae2-4e84-8b34-c73af78817a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.436405 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74eb6cb-8ae2-4e84-8b34-c73af78817a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.436685 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74eb6cb-8ae2-4e84-8b34-c73af78817a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.437377 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74eb6cb-8ae2-4e84-8b34-c73af78817a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:12.439373 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74eb6cb-8ae2-4e84-8b34-c73af78817a4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:12.461843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1296/2570] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:12:12 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:12:12.463493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df67a8-b4f2-401c-91ca-6585c252397b service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:12.473405 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df67a8-b4f2-401c-91ca-6585c252397b service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:12.479032 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc4557-2145-41f3-a7f2-d47c6f2f05fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:12.479708 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc4557-2145-41f3-a7f2-d47c6f2f05fa admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.479996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc4557-2145-41f3-a7f2-d47c6f2f05fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.480279 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc4557-2145-41f3-a7f2-d47c6f2f05fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.481012 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc4557-2145-41f3-a7f2-d47c6f2f05fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:12.482651 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc4557-2145-41f3-a7f2-d47c6f2f05fa admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:12.493706 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1275/2571] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:12.498584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74eb6cb-8ae2-4e84-8b34-c73af78817a4 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:12.509142 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74eb6cb-8ae2-4e84-8b34-c73af78817a4 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:12.513850 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e7a0c-09db-4021-93d5-ae38f12d7c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:12.514780 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e7a0c-09db-4021-93d5-ae38f12d7c85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.515069 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e7a0c-09db-4021-93d5-ae38f12d7c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.515350 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e7a0c-09db-4021-93d5-ae38f12d7c85 admin admin] PATH_INFO: `/v3/projects/b4935a5e731a4840bb246856af226259/users/bc39f504747b419da092769939d067a0/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.533466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e7a0c-09db-4021-93d5-ae38f12d7c85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4935a5e731a4840bb246856af226259, user_id=bc39f504747b419da092769939d067a0, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:12.533791 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e7a0c-09db-4021-93d5-ae38f12d7c85 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:12.554973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1297/2572] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:12:12 2022] PUT /identity/v3/projects/b4935a5e731a4840bb246856af226259/users/bc39f504747b419da092769939d067a0/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:12:12.559248 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc4557-2145-41f3-a7f2-d47c6f2f05fa admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:12.569379 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc4557-2145-41f3-a7f2-d47c6f2f05fa admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:12.619198 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:12.633265 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6d14a-6397-48e6-93fa-7e12cd830dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:12.633942 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6d14a-6397-48e6-93fa-7e12cd830dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.634252 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6d14a-6397-48e6-93fa-7e12cd830dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.634533 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6d14a-6397-48e6-93fa-7e12cd830dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.635261 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6d14a-6397-48e6-93fa-7e12cd830dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:12.636893 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6d14a-6397-48e6-93fa-7e12cd830dc0 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:12.647107 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1276/2573] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:12.651447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e7a0c-09db-4021-93d5-ae38f12d7c85 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:12.665005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e7a0c-09db-4021-93d5-ae38f12d7c85 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:12.666212 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6d14a-6397-48e6-93fa-7e12cd830dc0 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:12.668105 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-6af6d14a-6397-48e6-93fa-7e12cd830dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:12.669994 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b704f-c016-44cf-b1de-d33f3a880c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:12.670966 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b704f-c016-44cf-b1de-d33f3a880c9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.671342 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b704f-c016-44cf-b1de-d33f3a880c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.671536 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b704f-c016-44cf-b1de-d33f3a880c9e admin admin] PATH_INFO: `/v3/projects/b4935a5e731a4840bb246856af226259/users/bc39f504747b419da092769939d067a0/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.674244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b704f-c016-44cf-b1de-d33f3a880c9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4935a5e731a4840bb246856af226259, user_id=bc39f504747b419da092769939d067a0, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:12.677675 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b704f-c016-44cf-b1de-d33f3a880c9e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:12.683717 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6d14a-6397-48e6-93fa-7e12cd830dc0 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:12.688926 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38b287-b5ae-4dec-9212-3b2b5eef4179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:12.689802 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38b287-b5ae-4dec-9212-3b2b5eef4179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.690264 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38b287-b5ae-4dec-9212-3b2b5eef4179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.690556 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38b287-b5ae-4dec-9212-3b2b5eef4179 admin admin] PATH_INFO: `/v3/projects/97dc670f096448f3b084123d6b10fd42` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.691766 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38b287-b5ae-4dec-9212-3b2b5eef4179 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97dc670f096448f3b084123d6b10fd42)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:12.693685 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38b287-b5ae-4dec-9212-3b2b5eef4179 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:12.705030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1298/2574] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:12:12 2022] PUT /identity/v3/projects/b4935a5e731a4840bb246856af226259/users/bc39f504747b419da092769939d067a0/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:12.715956 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b6893-3c1b-464a-ad25-38947b27837a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.716580 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b6893-3c1b-464a-ad25-38947b27837a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.716580 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b6893-3c1b-464a-ad25-38947b27837a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.744472 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3e3b6893-3c1b-464a-ad25-38947b27837a None None] MFA Rules not processed for user `bc39f504747b419da092769939d067a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:12.745037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-4f38b287-b5ae-4dec-9212-3b2b5eef4179 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97dc670f096448f3b084123d6b10fd42', 'request_id': 'req-4f38b287-b5ae-4dec-9212-3b2b5eef4179'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:12.745737 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3e3b6893-3c1b-464a-ad25-38947b27837a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:12.753440 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-4f38b287-b5ae-4dec-9212-3b2b5eef4179 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97dc670f096448f3b084123d6b10fd42', 'request_id': 'req-4f38b287-b5ae-4dec-9212-3b2b5eef4179'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:12.755298 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:12.761608 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-4f38b287-b5ae-4dec-9212-3b2b5eef4179 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97dc670f096448f3b084123d6b10fd42', 'request_id': 'req-4f38b287-b5ae-4dec-9212-3b2b5eef4179'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:12.775728 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-4f38b287-b5ae-4dec-9212-3b2b5eef4179 admin admin] The token cache is being invalidate because project 97dc670f096448f3b084123d6b10fd42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:12.776105 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-4f38b287-b5ae-4dec-9212-3b2b5eef4179 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:12.776856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-4f38b287-b5ae-4dec-9212-3b2b5eef4179 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:12.778819 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1277/2575] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:12:12 2022] DELETE /identity/v3/projects/97dc670f096448f3b084123d6b10fd42 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:12:12.782850 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38b287-b5ae-4dec-9212-3b2b5eef4179 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:12.784667 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-4f38b287-b5ae-4dec-9212-3b2b5eef4179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:12.824239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1299/2576] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:12 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:12:12.834978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b6893-3c1b-464a-ad25-38947b27837a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:12.836993 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3e3b6893-3c1b-464a-ad25-38947b27837a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:12.852628 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38b287-b5ae-4dec-9212-3b2b5eef4179 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:12.857273 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c696293-362a-4dca-b00a-277f3deca28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:12.858151 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c696293-362a-4dca-b00a-277f3deca28d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.858465 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c696293-362a-4dca-b00a-277f3deca28d admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.858740 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c696293-362a-4dca-b00a-277f3deca28d admin admin] PATH_INFO: `/v3/users/715c24e4467f495883b374c297c58e33` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.859924 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c696293-362a-4dca-b00a-277f3deca28d admin admin] RBAC: Authorizing `identity:delete_user(user_id=715c24e4467f495883b374c297c58e33)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:12.861834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c696293-362a-4dca-b00a-277f3deca28d admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:12.899403 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b6893-3c1b-464a-ad25-38947b27837a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:12.903055 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992348b-d7a7-46ce-93d4-8d6775c7fb05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:12.903953 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992348b-d7a7-46ce-93d4-8d6775c7fb05 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:12.904234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992348b-d7a7-46ce-93d4-8d6775c7fb05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:12.904506 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992348b-d7a7-46ce-93d4-8d6775c7fb05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:12.906362 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a992348b-d7a7-46ce-93d4-8d6775c7fb05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:12.910718 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6c696293-362a-4dca-b00a-277f3deca28d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '715c24e4467f495883b374c297c58e33'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:12.919183 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6c696293-362a-4dca-b00a-277f3deca28d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '715c24e4467f495883b374c297c58e33'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:12.919928 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6c696293-362a-4dca-b00a-277f3deca28d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '715c24e4467f495883b374c297c58e33'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:12.920502 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6c696293-362a-4dca-b00a-277f3deca28d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '715c24e4467f495883b374c297c58e33'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:12.929349 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6c696293-362a-4dca-b00a-277f3deca28d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '715c24e4467f495883b374c297c58e33'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:12.935478 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:12.954137 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1278/2577] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:12:12 2022] DELETE /identity/v3/users/715c24e4467f495883b374c297c58e33 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:12:12.980679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992348b-d7a7-46ce-93d4-8d6775c7fb05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:12.982392 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992348b-d7a7-46ce-93d4-8d6775c7fb05 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:12.993566 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1300/2578] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:12:13.255295 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c696293-362a-4dca-b00a-277f3deca28d admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:13.256868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-6c696293-362a-4dca-b00a-277f3deca28d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:13.341745 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c696293-362a-4dca-b00a-277f3deca28d admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:13.347082 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7097b7-3695-4f5d-bf41-25dda2fa9151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:13.348025 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7097b7-3695-4f5d-bf41-25dda2fa9151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:13.348289 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7097b7-3695-4f5d-bf41-25dda2fa9151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:13.348514 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7097b7-3695-4f5d-bf41-25dda2fa9151 admin admin] PATH_INFO: `/v3/projects/c87ba2b7bebf40ff91474eb0f2b01e81` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:13.349730 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7097b7-3695-4f5d-bf41-25dda2fa9151 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c87ba2b7bebf40ff91474eb0f2b01e81)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:13.351739 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7097b7-3695-4f5d-bf41-25dda2fa9151 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:13.401343 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-bb7097b7-3695-4f5d-bf41-25dda2fa9151 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c87ba2b7bebf40ff91474eb0f2b01e81', 'request_id': 'req-bb7097b7-3695-4f5d-bf41-25dda2fa9151'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:13.410067 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-bb7097b7-3695-4f5d-bf41-25dda2fa9151 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c87ba2b7bebf40ff91474eb0f2b01e81', 'request_id': 'req-bb7097b7-3695-4f5d-bf41-25dda2fa9151'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:13.415966 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-bb7097b7-3695-4f5d-bf41-25dda2fa9151 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c87ba2b7bebf40ff91474eb0f2b01e81', 'request_id': 'req-bb7097b7-3695-4f5d-bf41-25dda2fa9151'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:13.427321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-bb7097b7-3695-4f5d-bf41-25dda2fa9151 admin admin] The token cache is being invalidate because project c87ba2b7bebf40ff91474eb0f2b01e81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:13.427610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-bb7097b7-3695-4f5d-bf41-25dda2fa9151 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:13.428245 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-bb7097b7-3695-4f5d-bf41-25dda2fa9151 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:13.429829 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1279/2579] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:12:13 2022] DELETE /identity/v3/projects/c87ba2b7bebf40ff91474eb0f2b01e81 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:13.434893 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8415a87-9b3e-4609-914d-a92b0914df37 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:13.435186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8415a87-9b3e-4609-914d-a92b0914df37 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:13.435475 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8415a87-9b3e-4609-914d-a92b0914df37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:13.454608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a8415a87-9b3e-4609-914d-a92b0914df37 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:13.455769 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a8415a87-9b3e-4609-914d-a92b0914df37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:13.519363 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1301/2580] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:13.527392 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04930e6-4801-43b0-9096-daaacd4eb38b None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:13.527887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04930e6-4801-43b0-9096-daaacd4eb38b None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:13.528610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04930e6-4801-43b0-9096-daaacd4eb38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:13.546917 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-a04930e6-4801-43b0-9096-daaacd4eb38b None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:13.548138 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-a04930e6-4801-43b0-9096-daaacd4eb38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:13.557015 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1280/2581] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:13.561447 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8415a87-9b3e-4609-914d-a92b0914df37 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:13.566225 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:13.573273 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8415a87-9b3e-4609-914d-a92b0914df37 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:13.577971 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6368bc-4383-497d-a52f-fb08fc96528c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:13.578766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6368bc-4383-497d-a52f-fb08fc96528c admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:13.578956 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6368bc-4383-497d-a52f-fb08fc96528c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:13.579346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6368bc-4383-497d-a52f-fb08fc96528c admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:13.579967 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6368bc-4383-497d-a52f-fb08fc96528c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:12:13.580277 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6368bc-4383-497d-a52f-fb08fc96528c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:13.581887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6368bc-4383-497d-a52f-fb08fc96528c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:13.594641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1302/2582] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:12:13.599048 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04930e6-4801-43b0-9096-daaacd4eb38b None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:13.608795 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04930e6-4801-43b0-9096-daaacd4eb38b None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:13.613589 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9fc6d-b86e-48f4-9be0-e13033d068b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:13.614282 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9fc6d-b86e-48f4-9be0-e13033d068b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:13.614572 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9fc6d-b86e-48f4-9be0-e13033d068b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:13.614851 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9fc6d-b86e-48f4-9be0-e13033d068b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:13.615558 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e9fc6d-b86e-48f4-9be0-e13033d068b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:13.617984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e9fc6d-b86e-48f4-9be0-e13033d068b0 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:13.643761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1281/2583] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:12:13 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:13.647997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6368bc-4383-497d-a52f-fb08fc96528c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:13.658316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6368bc-4383-497d-a52f-fb08fc96528c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:13.715881 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f87edd-e824-4467-9f82-5690173837e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:13.716556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f87edd-e824-4467-9f82-5690173837e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:13.717103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f87edd-e824-4467-9f82-5690173837e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:13.717103 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f87edd-e824-4467-9f82-5690173837e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:13.717768 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f87edd-e824-4467-9f82-5690173837e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:13.719753 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f87edd-e824-4467-9f82-5690173837e8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:13.740075 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1303/2584] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:12:13 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:13.744821 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9fc6d-b86e-48f4-9be0-e13033d068b0 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:13.755011 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9fc6d-b86e-48f4-9be0-e13033d068b0 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:13.760555 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d355cb2-1e08-4aed-b4dd-2ac8115cafe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:13.761215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d355cb2-1e08-4aed-b4dd-2ac8115cafe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:13.761505 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d355cb2-1e08-4aed-b4dd-2ac8115cafe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:13.761782 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d355cb2-1e08-4aed-b4dd-2ac8115cafe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:13.762551 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d355cb2-1e08-4aed-b4dd-2ac8115cafe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:13.764150 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d355cb2-1e08-4aed-b4dd-2ac8115cafe7 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:13.774648 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1282/2585] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:13.779382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f87edd-e824-4467-9f82-5690173837e8 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:13.790638 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f87edd-e824-4467-9f82-5690173837e8 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:13.795400 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11b382-af99-4bf7-bc46-8491d20a167f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:13.796062 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11b382-af99-4bf7-bc46-8491d20a167f admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:13.796344 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11b382-af99-4bf7-bc46-8491d20a167f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:13.796648 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11b382-af99-4bf7-bc46-8491d20a167f admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:13.797367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11b382-af99-4bf7-bc46-8491d20a167f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:13.799002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11b382-af99-4bf7-bc46-8491d20a167f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:13.810752 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1304/2586] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:13.815440 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d355cb2-1e08-4aed-b4dd-2ac8115cafe7 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:13.826849 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d355cb2-1e08-4aed-b4dd-2ac8115cafe7 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:13.831432 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe7161-3ab5-47a5-9dc1-d1c69fe784c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:13.832321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe7161-3ab5-47a5-9dc1-d1c69fe784c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:13.832601 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe7161-3ab5-47a5-9dc1-d1c69fe784c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:13.832873 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe7161-3ab5-47a5-9dc1-d1c69fe784c8 admin admin] PATH_INFO: `/v3/projects/fd60abab162f403ba5cbe07ae3ba4e7a/users/1a1b4b15047d42849249c300a7715bff/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:13.847938 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe7161-3ab5-47a5-9dc1-d1c69fe784c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd60abab162f403ba5cbe07ae3ba4e7a, user_id=1a1b4b15047d42849249c300a7715bff, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:13.850619 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe7161-3ab5-47a5-9dc1-d1c69fe784c8 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:13.864095 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1283/2587] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:12:13 2022] PUT /identity/v3/projects/fd60abab162f403ba5cbe07ae3ba4e7a/users/1a1b4b15047d42849249c300a7715bff/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:13.875998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0789c-3932-487a-991c-4ceb270adaf7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:13.876297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0789c-3932-487a-991c-4ceb270adaf7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:13.876613 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0789c-3932-487a-991c-4ceb270adaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:13.903058 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-56e0789c-3932-487a-991c-4ceb270adaf7 None None] MFA Rules not processed for user `1a1b4b15047d42849249c300a7715bff`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:13.904537 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-56e0789c-3932-487a-991c-4ceb270adaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:13.969074 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1305/2588] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:13.973585 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe7161-3ab5-47a5-9dc1-d1c69fe784c8 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:13.987835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:14.040165 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe7161-3ab5-47a5-9dc1-d1c69fe784c8 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:14.044844 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04c578-3af4-4851-bd9a-49a57ff17958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:14.045504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04c578-3af4-4851-bd9a-49a57ff17958 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:14.045791 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04c578-3af4-4851-bd9a-49a57ff17958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:14.046071 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04c578-3af4-4851-bd9a-49a57ff17958 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:14.046793 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04c578-3af4-4851-bd9a-49a57ff17958 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:14.048696 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04c578-3af4-4851-bd9a-49a57ff17958 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:14.076267 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1284/2589] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:12:13 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:14.080256 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0789c-3932-487a-991c-4ceb270adaf7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:14.091736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0789c-3932-487a-991c-4ceb270adaf7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:14.147811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:14.155241 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfeafe6-3349-4aad-b246-1e36e7acf51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:14.156036 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfeafe6-3349-4aad-b246-1e36e7acf51d admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:14.156338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfeafe6-3349-4aad-b246-1e36e7acf51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:14.156879 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfeafe6-3349-4aad-b246-1e36e7acf51d admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:14.157641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfeafe6-3349-4aad-b246-1e36e7acf51d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:14.159747 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfeafe6-3349-4aad-b246-1e36e7acf51d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:14.181707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1306/2590] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:12:14 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:14.185834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04c578-3af4-4851-bd9a-49a57ff17958 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:14.196493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04c578-3af4-4851-bd9a-49a57ff17958 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:14.201203 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366b597-98e4-4a37-99e1-c598c8f0207f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:14.201882 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366b597-98e4-4a37-99e1-c598c8f0207f admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:14.202171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366b597-98e4-4a37-99e1-c598c8f0207f admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:14.202486 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366b597-98e4-4a37-99e1-c598c8f0207f admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:14.203283 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366b597-98e4-4a37-99e1-c598c8f0207f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:14.204889 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366b597-98e4-4a37-99e1-c598c8f0207f admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:14.217055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1285/2591] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:12:14.220757 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfeafe6-3349-4aad-b246-1e36e7acf51d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:14.233013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfeafe6-3349-4aad-b246-1e36e7acf51d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:14.237783 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2dcd3-7778-4b13-afa2-6795c18d4006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:14.238493 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2dcd3-7778-4b13-afa2-6795c18d4006 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:14.238783 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2dcd3-7778-4b13-afa2-6795c18d4006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:14.239065 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2dcd3-7778-4b13-afa2-6795c18d4006 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:14.239797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2dcd3-7778-4b13-afa2-6795c18d4006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:14.241582 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2dcd3-7778-4b13-afa2-6795c18d4006 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:14.252231 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1307/2592] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:12:14.256302 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366b597-98e4-4a37-99e1-c598c8f0207f admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:14.266504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366b597-98e4-4a37-99e1-c598c8f0207f admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:14.271943 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9b600-3da4-4ad6-bc31-c404d616772a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:14.273419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9b600-3da4-4ad6-bc31-c404d616772a admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:14.273711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9b600-3da4-4ad6-bc31-c404d616772a admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:14.273994 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9b600-3da4-4ad6-bc31-c404d616772a admin admin] PATH_INFO: `/v3/projects/cea5877ed4654d60b12e400d79ed318d/users/37eea7c65c144365bac8702d703b556f/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:14.291256 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9b600-3da4-4ad6-bc31-c404d616772a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cea5877ed4654d60b12e400d79ed318d, user_id=37eea7c65c144365bac8702d703b556f, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:14.293918 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9b600-3da4-4ad6-bc31-c404d616772a admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:14.307232 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1286/2593] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:12:14 2022] PUT /identity/v3/projects/cea5877ed4654d60b12e400d79ed318d/users/37eea7c65c144365bac8702d703b556f/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:14.322674 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc434c4-2ba7-4386-b785-06f854f8ad57 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:14.323409 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc434c4-2ba7-4386-b785-06f854f8ad57 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:14.323409 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc434c4-2ba7-4386-b785-06f854f8ad57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:14.354865 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-7bc434c4-2ba7-4386-b785-06f854f8ad57 None None] MFA Rules not processed for user `37eea7c65c144365bac8702d703b556f`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:14.356206 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7bc434c4-2ba7-4386-b785-06f854f8ad57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:14.437145 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1308/2594] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:14.442937 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9b600-3da4-4ad6-bc31-c404d616772a admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:14.510872 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9b600-3da4-4ad6-bc31-c404d616772a admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:14.515878 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7c04d-304c-47ae-97ba-0161a75e158f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:14.516574 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7c04d-304c-47ae-97ba-0161a75e158f admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:14.516855 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7c04d-304c-47ae-97ba-0161a75e158f admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:14.517291 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7c04d-304c-47ae-97ba-0161a75e158f admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:14.517988 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7c04d-304c-47ae-97ba-0161a75e158f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:14.519921 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7c04d-304c-47ae-97ba-0161a75e158f admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:14.547505 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1287/2595] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:12:14 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:14.551291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc434c4-2ba7-4386-b785-06f854f8ad57 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:14.565741 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc434c4-2ba7-4386-b785-06f854f8ad57 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:14.627341 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73a500-2fda-4379-8a0d-d106367b720c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:14.627916 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73a500-2fda-4379-8a0d-d106367b720c admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:14.628199 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73a500-2fda-4379-8a0d-d106367b720c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:14.628473 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73a500-2fda-4379-8a0d-d106367b720c admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:14.629156 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a73a500-2fda-4379-8a0d-d106367b720c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:14.631116 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a73a500-2fda-4379-8a0d-d106367b720c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:14.653846 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1309/2596] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:12:14 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:14.658001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7c04d-304c-47ae-97ba-0161a75e158f admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:14.668942 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7c04d-304c-47ae-97ba-0161a75e158f admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:14.673773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71373743-c7cd-4ff3-9477-0119e5009412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:14.674482 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71373743-c7cd-4ff3-9477-0119e5009412 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:14.674772 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71373743-c7cd-4ff3-9477-0119e5009412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:14.675053 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71373743-c7cd-4ff3-9477-0119e5009412 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:14.675810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71373743-c7cd-4ff3-9477-0119e5009412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:14.677394 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71373743-c7cd-4ff3-9477-0119e5009412 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:14.688816 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1288/2597] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:14.693230 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73a500-2fda-4379-8a0d-d106367b720c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:14.704260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73a500-2fda-4379-8a0d-d106367b720c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:14.709251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa31d6f1-0435-4e31-a42c-d57bba166c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:14.710938 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31d6f1-0435-4e31-a42c-d57bba166c1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:14.711167 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31d6f1-0435-4e31-a42c-d57bba166c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:14.711527 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31d6f1-0435-4e31-a42c-d57bba166c1c admin admin] PATH_INFO: `/v3/projects/11f11d498f7b4a149e363ed5dfc1a28b/users/0293bb8ce55146de837bc39fb1a88d3f/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:14.727895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa31d6f1-0435-4e31-a42c-d57bba166c1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=11f11d498f7b4a149e363ed5dfc1a28b, user_id=0293bb8ce55146de837bc39fb1a88d3f, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:14.730860 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa31d6f1-0435-4e31-a42c-d57bba166c1c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:14.743479 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1310/2598] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:12:14 2022] PUT /identity/v3/projects/11f11d498f7b4a149e363ed5dfc1a28b/users/0293bb8ce55146de837bc39fb1a88d3f/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:14.747444 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71373743-c7cd-4ff3-9477-0119e5009412 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:14.757996 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71373743-c7cd-4ff3-9477-0119e5009412 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:14.797828 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:14.824274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a7303-bf5c-4677-bffb-fa0aac68ded7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:14.825179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a7303-bf5c-4677-bffb-fa0aac68ded7 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:14.825606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a7303-bf5c-4677-bffb-fa0aac68ded7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:14.825931 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a7303-bf5c-4677-bffb-fa0aac68ded7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:14.826765 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a7303-bf5c-4677-bffb-fa0aac68ded7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:14.828445 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a7303-bf5c-4677-bffb-fa0aac68ded7 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:14.840550 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1289/2599] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:14.844844 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa31d6f1-0435-4e31-a42c-d57bba166c1c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:14.855292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa31d6f1-0435-4e31-a42c-d57bba166c1c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:14.859890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca46a8e-d1d6-414b-baf6-4957f4323ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:14.860762 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca46a8e-d1d6-414b-baf6-4957f4323ee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:14.861046 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca46a8e-d1d6-414b-baf6-4957f4323ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:14.861319 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca46a8e-d1d6-414b-baf6-4957f4323ee5 admin admin] PATH_INFO: `/v3/projects/11f11d498f7b4a149e363ed5dfc1a28b/users/0293bb8ce55146de837bc39fb1a88d3f/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:14.863542 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca46a8e-d1d6-414b-baf6-4957f4323ee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11f11d498f7b4a149e363ed5dfc1a28b, user_id=0293bb8ce55146de837bc39fb1a88d3f, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:14.866201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca46a8e-d1d6-414b-baf6-4957f4323ee5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:14.877478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1311/2600] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:12:14 2022] PUT /identity/v3/projects/11f11d498f7b4a149e363ed5dfc1a28b/users/0293bb8ce55146de837bc39fb1a88d3f/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:14.888308 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06830a-1797-4505-8f0e-a7409b94cc1f None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:14.888605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06830a-1797-4505-8f0e-a7409b94cc1f None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:14.889057 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06830a-1797-4505-8f0e-a7409b94cc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:14.917338 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-aa06830a-1797-4505-8f0e-a7409b94cc1f None None] MFA Rules not processed for user `0293bb8ce55146de837bc39fb1a88d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:14.918587 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-aa06830a-1797-4505-8f0e-a7409b94cc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:14.988246 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1290/2601] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:14 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:12:15.000678 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca46a8e-d1d6-414b-baf6-4957f4323ee5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:15.002479 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6ca46a8e-d1d6-414b-baf6-4957f4323ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:15.024315 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:15.063336 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca46a8e-d1d6-414b-baf6-4957f4323ee5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:15.067021 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e8b68-f7d1-4f67-bef3-9580c794a947 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:15.067666 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e8b68-f7d1-4f67-bef3-9580c794a947 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:15.067950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e8b68-f7d1-4f67-bef3-9580c794a947 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:15.068226 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e8b68-f7d1-4f67-bef3-9580c794a947 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:15.078004 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9e8b68-f7d1-4f67-bef3-9580c794a947 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:15.079594 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9e8b68-f7d1-4f67-bef3-9580c794a947 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:15.089337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1312/2602] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:14 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:15.131640 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06830a-1797-4505-8f0e-a7409b94cc1f None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:15.133615 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-aa06830a-1797-4505-8f0e-a7409b94cc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:15.145718 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06830a-1797-4505-8f0e-a7409b94cc1f None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:15.150116 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd52e3f-0e6b-4928-8170-eb51b6530a61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:15.150810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd52e3f-0e6b-4928-8170-eb51b6530a61 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:15.151220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd52e3f-0e6b-4928-8170-eb51b6530a61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:15.151370 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd52e3f-0e6b-4928-8170-eb51b6530a61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:15.208117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd52e3f-0e6b-4928-8170-eb51b6530a61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:15.209561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd52e3f-0e6b-4928-8170-eb51b6530a61 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:15.219742 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1291/2603] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:15.566512 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e8b68-f7d1-4f67-bef3-9580c794a947 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:15.568808 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5b9e8b68-f7d1-4f67-bef3-9580c794a947 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:15.643384 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e8b68-f7d1-4f67-bef3-9580c794a947 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:15.648691 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392d55d-9321-4e34-a048-c78589f96ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:15.649598 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392d55d-9321-4e34-a048-c78589f96ef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:15.649878 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392d55d-9321-4e34-a048-c78589f96ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:15.650152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392d55d-9321-4e34-a048-c78589f96ef9 admin admin] PATH_INFO: `/v3/users/bc75518bc7634a41b2dec8fd30b22ac8` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:15.651416 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9392d55d-9321-4e34-a048-c78589f96ef9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc75518bc7634a41b2dec8fd30b22ac8)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:15.653264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9392d55d-9321-4e34-a048-c78589f96ef9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:15.703135 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9392d55d-9321-4e34-a048-c78589f96ef9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc75518bc7634a41b2dec8fd30b22ac8'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:15.710964 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9392d55d-9321-4e34-a048-c78589f96ef9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc75518bc7634a41b2dec8fd30b22ac8'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:15.722877 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9392d55d-9321-4e34-a048-c78589f96ef9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc75518bc7634a41b2dec8fd30b22ac8'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:15.723529 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9392d55d-9321-4e34-a048-c78589f96ef9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc75518bc7634a41b2dec8fd30b22ac8'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:15.724140 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9392d55d-9321-4e34-a048-c78589f96ef9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc75518bc7634a41b2dec8fd30b22ac8'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:15.736976 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1313/2604] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:12:15 2022] DELETE /identity/v3/users/bc75518bc7634a41b2dec8fd30b22ac8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:12:15.741546 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d0768-830d-4eb1-9fd6-ed19ff87a8ae None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:15.741965 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d0768-830d-4eb1-9fd6-ed19ff87a8ae None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:15.742120 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d0768-830d-4eb1-9fd6-ed19ff87a8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:15.759603 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-ef5d0768-830d-4eb1-9fd6-ed19ff87a8ae None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:15.760868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-ef5d0768-830d-4eb1-9fd6-ed19ff87a8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:15.813184 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:15.823276 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1292/2605] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:15.832872 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392d55d-9321-4e34-a048-c78589f96ef9 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:15.834494 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9392d55d-9321-4e34-a048-c78589f96ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:15.879009 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:15.893768 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392d55d-9321-4e34-a048-c78589f96ef9 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:15.897484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c3299-c25e-4db6-aded-d07a721292a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:15.898688 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c3299-c25e-4db6-aded-d07a721292a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:15.899056 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c3299-c25e-4db6-aded-d07a721292a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:15.899254 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c3299-c25e-4db6-aded-d07a721292a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:15.909320 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c3299-c25e-4db6-aded-d07a721292a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:15.910820 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c3299-c25e-4db6-aded-d07a721292a9 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:15.921073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1314/2606] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:16.202202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d0768-830d-4eb1-9fd6-ed19ff87a8ae None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:16.203669 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-ef5d0768-830d-4eb1-9fd6-ed19ff87a8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:16.216587 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d0768-830d-4eb1-9fd6-ed19ff87a8ae None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:16.221330 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e648690-6dde-416c-8dea-d859f1cafe7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:16.222258 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e648690-6dde-416c-8dea-d859f1cafe7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:16.222552 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e648690-6dde-416c-8dea-d859f1cafe7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:16.222837 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e648690-6dde-416c-8dea-d859f1cafe7b admin admin] PATH_INFO: `/v3/projects/751c5807fa834ced9a9dd32ff99da2f6` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:16.224045 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e648690-6dde-416c-8dea-d859f1cafe7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=751c5807fa834ced9a9dd32ff99da2f6)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:16.226584 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e648690-6dde-416c-8dea-d859f1cafe7b admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:16.275802 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6e648690-6dde-416c-8dea-d859f1cafe7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '751c5807fa834ced9a9dd32ff99da2f6', 'request_id': 'req-6e648690-6dde-416c-8dea-d859f1cafe7b'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:16.282912 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6e648690-6dde-416c-8dea-d859f1cafe7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '751c5807fa834ced9a9dd32ff99da2f6', 'request_id': 'req-6e648690-6dde-416c-8dea-d859f1cafe7b'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:16.289050 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6e648690-6dde-416c-8dea-d859f1cafe7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '751c5807fa834ced9a9dd32ff99da2f6', 'request_id': 'req-6e648690-6dde-416c-8dea-d859f1cafe7b'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:16.300139 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6e648690-6dde-416c-8dea-d859f1cafe7b admin admin] The token cache is being invalidate because project 751c5807fa834ced9a9dd32ff99da2f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:16.300426 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6e648690-6dde-416c-8dea-d859f1cafe7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:16.301063 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6e648690-6dde-416c-8dea-d859f1cafe7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:16.302645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1293/2607] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:12:16 2022] DELETE /identity/v3/projects/751c5807fa834ced9a9dd32ff99da2f6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:16.307375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c3299-c25e-4db6-aded-d07a721292a9 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:16.317306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c3299-c25e-4db6-aded-d07a721292a9 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:16.375133 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fe0ab-b846-4166-9d89-b42edced5f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:16.375998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fe0ab-b846-4166-9d89-b42edced5f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:16.376282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fe0ab-b846-4166-9d89-b42edced5f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:16.376561 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fe0ab-b846-4166-9d89-b42edced5f07 admin admin] PATH_INFO: `/v3/users/bc39f504747b419da092769939d067a0` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:16.377739 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410fe0ab-b846-4166-9d89-b42edced5f07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc39f504747b419da092769939d067a0)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:16.379679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410fe0ab-b846-4166-9d89-b42edced5f07 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:16.425720 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-410fe0ab-b846-4166-9d89-b42edced5f07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc39f504747b419da092769939d067a0'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:16.435584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-410fe0ab-b846-4166-9d89-b42edced5f07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc39f504747b419da092769939d067a0'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:16.448042 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-410fe0ab-b846-4166-9d89-b42edced5f07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc39f504747b419da092769939d067a0'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:16.448701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-410fe0ab-b846-4166-9d89-b42edced5f07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc39f504747b419da092769939d067a0'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:16.449308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-410fe0ab-b846-4166-9d89-b42edced5f07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc39f504747b419da092769939d067a0'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:16.461871 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1315/2608] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:12:16 2022] DELETE /identity/v3/users/bc39f504747b419da092769939d067a0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:16.824831 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e648690-6dde-416c-8dea-d859f1cafe7b admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:16.826310 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-6e648690-6dde-416c-8dea-d859f1cafe7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:16.842306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e648690-6dde-416c-8dea-d859f1cafe7b admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:16.906674 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9db7c0-67b7-4976-bea6-6e92677bc613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:16.907721 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9db7c0-67b7-4976-bea6-6e92677bc613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:16.908309 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9db7c0-67b7-4976-bea6-6e92677bc613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:16.908592 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9db7c0-67b7-4976-bea6-6e92677bc613 admin admin] PATH_INFO: `/v3/projects/b4935a5e731a4840bb246856af226259` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:16.909797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9db7c0-67b7-4976-bea6-6e92677bc613 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4935a5e731a4840bb246856af226259)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:16.911804 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9db7c0-67b7-4976-bea6-6e92677bc613 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:16.968580 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-5b9db7c0-67b7-4976-bea6-6e92677bc613 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4935a5e731a4840bb246856af226259', 'request_id': 'req-5b9db7c0-67b7-4976-bea6-6e92677bc613'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:16.977782 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-5b9db7c0-67b7-4976-bea6-6e92677bc613 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4935a5e731a4840bb246856af226259', 'request_id': 'req-5b9db7c0-67b7-4976-bea6-6e92677bc613'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:16.984439 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-5b9db7c0-67b7-4976-bea6-6e92677bc613 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4935a5e731a4840bb246856af226259', 'request_id': 'req-5b9db7c0-67b7-4976-bea6-6e92677bc613'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:16.996628 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-5b9db7c0-67b7-4976-bea6-6e92677bc613 admin admin] The token cache is being invalidate because project b4935a5e731a4840bb246856af226259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:16.996918 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-5b9db7c0-67b7-4976-bea6-6e92677bc613 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:16.997683 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-5b9db7c0-67b7-4976-bea6-6e92677bc613 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:17.000251 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1294/2609] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:12:16 2022] DELETE /identity/v3/projects/b4935a5e731a4840bb246856af226259 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:17.005615 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7d27d-7e75-4a6d-abb7-3c596de2bbd0 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:17.006102 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7d27d-7e75-4a6d-abb7-3c596de2bbd0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:17.006429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7d27d-7e75-4a6d-abb7-3c596de2bbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:17.026374 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4db7d27d-7e75-4a6d-abb7-3c596de2bbd0 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:17.027741 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4db7d27d-7e75-4a6d-abb7-3c596de2bbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:17.091260 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1316/2610] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:17.099150 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1a06e-de2c-4cee-bbec-78f09f876b28 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:17.099443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1a06e-de2c-4cee-bbec-78f09f876b28 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:17.099757 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1a06e-de2c-4cee-bbec-78f09f876b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:17.119908 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-4af1a06e-de2c-4cee-bbec-78f09f876b28 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:17.121246 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-4af1a06e-de2c-4cee-bbec-78f09f876b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:17.130234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1295/2611] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:17.134444 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7d27d-7e75-4a6d-abb7-3c596de2bbd0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:17.139355 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:17.145866 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7d27d-7e75-4a6d-abb7-3c596de2bbd0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:17.152947 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f6424-8c51-4e6b-90ed-277e0809aeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:17.153867 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f6424-8c51-4e6b-90ed-277e0809aeb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:17.154148 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f6424-8c51-4e6b-90ed-277e0809aeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:17.154452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f6424-8c51-4e6b-90ed-277e0809aeb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:17.155177 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f6424-8c51-4e6b-90ed-277e0809aeb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:12:17.155485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f6424-8c51-4e6b-90ed-277e0809aeb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:17.157101 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f6424-8c51-4e6b-90ed-277e0809aeb7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:17.169500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1317/2612] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:12:17.173976 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1a06e-de2c-4cee-bbec-78f09f876b28 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:17.180871 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:17.184425 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1a06e-de2c-4cee-bbec-78f09f876b28 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:17.190324 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd284597-dee0-44e5-a2e3-c02881fd94f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:17.190956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd284597-dee0-44e5-a2e3-c02881fd94f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:17.191253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd284597-dee0-44e5-a2e3-c02881fd94f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:17.191578 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd284597-dee0-44e5-a2e3-c02881fd94f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:17.192300 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd284597-dee0-44e5-a2e3-c02881fd94f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:17.194263 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd284597-dee0-44e5-a2e3-c02881fd94f3 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:17.226994 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1296/2613] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:12:17 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:17.231749 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f6424-8c51-4e6b-90ed-277e0809aeb7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:17.242965 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f6424-8c51-4e6b-90ed-277e0809aeb7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:17.307465 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e072b4-d31d-40eb-b407-059956a303b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:17.308169 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e072b4-d31d-40eb-b407-059956a303b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:17.308506 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e072b4-d31d-40eb-b407-059956a303b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:17.308718 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e072b4-d31d-40eb-b407-059956a303b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:17.309402 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e072b4-d31d-40eb-b407-059956a303b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:17.311581 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e072b4-d31d-40eb-b407-059956a303b4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:17.334239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1318/2614] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:12:17 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:12:17.336600 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd284597-dee0-44e5-a2e3-c02881fd94f3 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:17.347250 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd284597-dee0-44e5-a2e3-c02881fd94f3 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:17.352558 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab3d6c-2303-4825-b87c-e247ba985f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:17.353015 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab3d6c-2303-4825-b87c-e247ba985f34 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:17.353300 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab3d6c-2303-4825-b87c-e247ba985f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:17.353577 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab3d6c-2303-4825-b87c-e247ba985f34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:17.354338 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab3d6c-2303-4825-b87c-e247ba985f34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:17.355946 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab3d6c-2303-4825-b87c-e247ba985f34 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:17.367620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1297/2615] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:17.371460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e072b4-d31d-40eb-b407-059956a303b4 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:17.381944 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e072b4-d31d-40eb-b407-059956a303b4 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:17.387415 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc21bf4-9008-4d7d-b63a-e98860898775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:17.388169 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc21bf4-9008-4d7d-b63a-e98860898775 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:17.388448 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc21bf4-9008-4d7d-b63a-e98860898775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:17.388950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc21bf4-9008-4d7d-b63a-e98860898775 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:17.389687 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc21bf4-9008-4d7d-b63a-e98860898775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:17.391325 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc21bf4-9008-4d7d-b63a-e98860898775 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:17.401718 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1319/2616] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:17.406214 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab3d6c-2303-4825-b87c-e247ba985f34 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:17.416042 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab3d6c-2303-4825-b87c-e247ba985f34 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:17.420671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063470e-3e71-42e5-93fc-a91aa3e8f45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:17.421535 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063470e-3e71-42e5-93fc-a91aa3e8f45b admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:17.421815 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063470e-3e71-42e5-93fc-a91aa3e8f45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:17.422091 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063470e-3e71-42e5-93fc-a91aa3e8f45b admin admin] PATH_INFO: `/v3/projects/6217142c97d54164a23b27dafacd083f/users/e98003818ed84c9b8e2301467edebdd6/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:17.437016 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063470e-3e71-42e5-93fc-a91aa3e8f45b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6217142c97d54164a23b27dafacd083f, user_id=e98003818ed84c9b8e2301467edebdd6, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:17.439733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063470e-3e71-42e5-93fc-a91aa3e8f45b admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:17.455856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1298/2617] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:12:17 2022] PUT /identity/v3/projects/6217142c97d54164a23b27dafacd083f/users/e98003818ed84c9b8e2301467edebdd6/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:12:17.459529 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6a10d-3481-4532-9585-8e6452e05432 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:17.459827 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6a10d-3481-4532-9585-8e6452e05432 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:17.460112 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6a10d-3481-4532-9585-8e6452e05432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:17.479806 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-24c6a10d-3481-4532-9585-8e6452e05432 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:17.481031 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-24c6a10d-3481-4532-9585-8e6452e05432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:17.540308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1320/2618] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:17.549858 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063470e-3e71-42e5-93fc-a91aa3e8f45b admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:17.551557 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-b063470e-3e71-42e5-93fc-a91aa3e8f45b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:17.611206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063470e-3e71-42e5-93fc-a91aa3e8f45b admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:17.614804 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72af0fb-60a1-45d9-9e1d-09b7a7ca12f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:17.615446 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72af0fb-60a1-45d9-9e1d-09b7a7ca12f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:17.615730 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72af0fb-60a1-45d9-9e1d-09b7a7ca12f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:17.616005 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72af0fb-60a1-45d9-9e1d-09b7a7ca12f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:17.625959 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72af0fb-60a1-45d9-9e1d-09b7a7ca12f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:17.627419 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72af0fb-60a1-45d9-9e1d-09b7a7ca12f0 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:17.636963 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1299/2619] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:18.173452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6a10d-3481-4532-9585-8e6452e05432 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:18.174962 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-24c6a10d-3481-4532-9585-8e6452e05432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:18.187822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6a10d-3481-4532-9585-8e6452e05432 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:18.194320 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e26e3b-26a9-4f30-b78e-ff6f05ccfe0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:18.195913 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e26e3b-26a9-4f30-b78e-ff6f05ccfe0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:18.196197 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e26e3b-26a9-4f30-b78e-ff6f05ccfe0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:18.196483 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e26e3b-26a9-4f30-b78e-ff6f05ccfe0e admin admin] PATH_INFO: `/v3/users/1a1b4b15047d42849249c300a7715bff` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:18.197694 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e26e3b-26a9-4f30-b78e-ff6f05ccfe0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a1b4b15047d42849249c300a7715bff)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:18.199662 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e26e3b-26a9-4f30-b78e-ff6f05ccfe0e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:18.259520 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d6e26e3b-26a9-4f30-b78e-ff6f05ccfe0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a1b4b15047d42849249c300a7715bff'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:18.270098 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d6e26e3b-26a9-4f30-b78e-ff6f05ccfe0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a1b4b15047d42849249c300a7715bff'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:18.279988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:18.286494 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d6e26e3b-26a9-4f30-b78e-ff6f05ccfe0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a1b4b15047d42849249c300a7715bff'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:18.287525 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d6e26e3b-26a9-4f30-b78e-ff6f05ccfe0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a1b4b15047d42849249c300a7715bff'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:18.287959 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d6e26e3b-26a9-4f30-b78e-ff6f05ccfe0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a1b4b15047d42849249c300a7715bff'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:18.302932 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1321/2620] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:12:18 2022] DELETE /identity/v3/users/1a1b4b15047d42849249c300a7715bff => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:18.308190 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e77c01-81fb-46a0-bf8f-5f3ca96206f4 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:18.308682 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e77c01-81fb-46a0-bf8f-5f3ca96206f4 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:18.308900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e77c01-81fb-46a0-bf8f-5f3ca96206f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:18.338081 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7765b1-62e0-44ae-8e93-a827daed5794 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:18.338530 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7765b1-62e0-44ae-8e93-a827daed5794 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:18.338905 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7765b1-62e0-44ae-8e93-a827daed5794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:18.356078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-86e77c01-81fb-46a0-bf8f-5f3ca96206f4 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:18.356078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-86e77c01-81fb-46a0-bf8f-5f3ca96206f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:18.379890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6e7765b1-62e0-44ae-8e93-a827daed5794 None None] MFA Rules not processed for user `e98003818ed84c9b8e2301467edebdd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:18.383306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6e7765b1-62e0-44ae-8e93-a827daed5794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:18.427190 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1300/2621] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:18.438596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e77c01-81fb-46a0-bf8f-5f3ca96206f4 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:18.440232 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-86e77c01-81fb-46a0-bf8f-5f3ca96206f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:18.462426 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1322/2622] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:18.472876 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7765b1-62e0-44ae-8e93-a827daed5794 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:18.474289 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6e7765b1-62e0-44ae-8e93-a827daed5794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:18.518726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e77c01-81fb-46a0-bf8f-5f3ca96206f4 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:18.526166 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11a10c-ae19-40a6-82cc-e56bdee3fc52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:18.526946 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11a10c-ae19-40a6-82cc-e56bdee3fc52 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:18.527491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11a10c-ae19-40a6-82cc-e56bdee3fc52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:18.527631 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11a10c-ae19-40a6-82cc-e56bdee3fc52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:18.534978 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:18.539053 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11a10c-ae19-40a6-82cc-e56bdee3fc52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:18.540832 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7765b1-62e0-44ae-8e93-a827daed5794 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:18.540967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11a10c-ae19-40a6-82cc-e56bdee3fc52 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:18.544564 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75d9d3-a39f-4705-bfcc-30c392122b68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:18.545222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75d9d3-a39f-4705-bfcc-30c392122b68 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:18.545507 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75d9d3-a39f-4705-bfcc-30c392122b68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:18.545800 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75d9d3-a39f-4705-bfcc-30c392122b68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:18.552449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1301/2623] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:12:18.554975 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75d9d3-a39f-4705-bfcc-30c392122b68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:18.556518 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75d9d3-a39f-4705-bfcc-30c392122b68 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:18.566989 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1323/2624] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:12:18.870213 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11a10c-ae19-40a6-82cc-e56bdee3fc52 service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:18.871683 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-1b11a10c-ae19-40a6-82cc-e56bdee3fc52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:18.893604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11a10c-ae19-40a6-82cc-e56bdee3fc52 service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:18.901532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdcf304-2ff1-4c72-a728-f74403c091f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:18.902973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdcf304-2ff1-4c72-a728-f74403c091f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:18.904001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdcf304-2ff1-4c72-a728-f74403c091f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:18.904609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdcf304-2ff1-4c72-a728-f74403c091f2 admin admin] PATH_INFO: `/v3/projects/fd60abab162f403ba5cbe07ae3ba4e7a` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:18.907048 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdcf304-2ff1-4c72-a728-f74403c091f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd60abab162f403ba5cbe07ae3ba4e7a)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:18.909992 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdcf304-2ff1-4c72-a728-f74403c091f2 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:18.967716 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2cdcf304-2ff1-4c72-a728-f74403c091f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd60abab162f403ba5cbe07ae3ba4e7a', 'request_id': 'req-2cdcf304-2ff1-4c72-a728-f74403c091f2'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:18.975940 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2cdcf304-2ff1-4c72-a728-f74403c091f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd60abab162f403ba5cbe07ae3ba4e7a', 'request_id': 'req-2cdcf304-2ff1-4c72-a728-f74403c091f2'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:18.981622 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2cdcf304-2ff1-4c72-a728-f74403c091f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd60abab162f403ba5cbe07ae3ba4e7a', 'request_id': 'req-2cdcf304-2ff1-4c72-a728-f74403c091f2'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:18.993462 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2cdcf304-2ff1-4c72-a728-f74403c091f2 admin admin] The token cache is being invalidate because project fd60abab162f403ba5cbe07ae3ba4e7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:18.993783 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2cdcf304-2ff1-4c72-a728-f74403c091f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:18.994453 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-2cdcf304-2ff1-4c72-a728-f74403c091f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:18.996016 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1302/2625] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:12:18 2022] DELETE /identity/v3/projects/fd60abab162f403ba5cbe07ae3ba4e7a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:19.000291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75d9d3-a39f-4705-bfcc-30c392122b68 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:19.012327 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75d9d3-a39f-4705-bfcc-30c392122b68 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:19.069398 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ba282-dfad-4161-8908-ba9a48b1b670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:19.070946 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ba282-dfad-4161-8908-ba9a48b1b670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:19.071239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ba282-dfad-4161-8908-ba9a48b1b670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:19.071519 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ba282-dfad-4161-8908-ba9a48b1b670 admin admin] PATH_INFO: `/v3/users/37eea7c65c144365bac8702d703b556f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:19.072733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ba282-dfad-4161-8908-ba9a48b1b670 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37eea7c65c144365bac8702d703b556f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:19.074703 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ba282-dfad-4161-8908-ba9a48b1b670 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:19.088559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:19.123634 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-152ba282-dfad-4161-8908-ba9a48b1b670 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37eea7c65c144365bac8702d703b556f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:19.137241 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-152ba282-dfad-4161-8908-ba9a48b1b670 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37eea7c65c144365bac8702d703b556f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:19.151344 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-152ba282-dfad-4161-8908-ba9a48b1b670 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37eea7c65c144365bac8702d703b556f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:19.152036 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-152ba282-dfad-4161-8908-ba9a48b1b670 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37eea7c65c144365bac8702d703b556f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:19.152637 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-152ba282-dfad-4161-8908-ba9a48b1b670 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37eea7c65c144365bac8702d703b556f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:19.166458 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1324/2626] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:12:18 2022] DELETE /identity/v3/users/37eea7c65c144365bac8702d703b556f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:19.521618 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdcf304-2ff1-4c72-a728-f74403c091f2 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:19.523641 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-2cdcf304-2ff1-4c72-a728-f74403c091f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:19.540572 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdcf304-2ff1-4c72-a728-f74403c091f2 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:19.599089 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675daf17-d09b-46ed-9ea7-3c983a0cb405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:19.599867 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675daf17-d09b-46ed-9ea7-3c983a0cb405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:19.600154 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675daf17-d09b-46ed-9ea7-3c983a0cb405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:19.600434 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675daf17-d09b-46ed-9ea7-3c983a0cb405 admin admin] PATH_INFO: `/v3/projects/cea5877ed4654d60b12e400d79ed318d` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:19.602753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675daf17-d09b-46ed-9ea7-3c983a0cb405 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cea5877ed4654d60b12e400d79ed318d)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:19.604675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675daf17-d09b-46ed-9ea7-3c983a0cb405 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:19.650550 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-675daf17-d09b-46ed-9ea7-3c983a0cb405 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cea5877ed4654d60b12e400d79ed318d', 'request_id': 'req-675daf17-d09b-46ed-9ea7-3c983a0cb405'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:19.658118 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-675daf17-d09b-46ed-9ea7-3c983a0cb405 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cea5877ed4654d60b12e400d79ed318d', 'request_id': 'req-675daf17-d09b-46ed-9ea7-3c983a0cb405'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:19.664305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-675daf17-d09b-46ed-9ea7-3c983a0cb405 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cea5877ed4654d60b12e400d79ed318d', 'request_id': 'req-675daf17-d09b-46ed-9ea7-3c983a0cb405'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:19.675480 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-675daf17-d09b-46ed-9ea7-3c983a0cb405 admin admin] The token cache is being invalidate because project cea5877ed4654d60b12e400d79ed318d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:19.675771 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-675daf17-d09b-46ed-9ea7-3c983a0cb405 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:19.676426 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-675daf17-d09b-46ed-9ea7-3c983a0cb405 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:19.678031 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1303/2627] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:12:19 2022] DELETE /identity/v3/projects/cea5877ed4654d60b12e400d79ed318d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:19.682219 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ba282-dfad-4161-8908-ba9a48b1b670 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:19.683733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-152ba282-dfad-4161-8908-ba9a48b1b670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:19.697327 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ba282-dfad-4161-8908-ba9a48b1b670 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:19.754720 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb543ee-dbbe-4059-9c0f-403dc37cf8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:19.755579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb543ee-dbbe-4059-9c0f-403dc37cf8b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:19.755863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb543ee-dbbe-4059-9c0f-403dc37cf8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:19.756138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb543ee-dbbe-4059-9c0f-403dc37cf8b3 admin admin] PATH_INFO: `/v3/users/0293bb8ce55146de837bc39fb1a88d3f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:19.757849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb543ee-dbbe-4059-9c0f-403dc37cf8b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0293bb8ce55146de837bc39fb1a88d3f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:19.759799 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb543ee-dbbe-4059-9c0f-403dc37cf8b3 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:19.813518 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2cb543ee-dbbe-4059-9c0f-403dc37cf8b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0293bb8ce55146de837bc39fb1a88d3f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:19.821686 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2cb543ee-dbbe-4059-9c0f-403dc37cf8b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0293bb8ce55146de837bc39fb1a88d3f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:19.833619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2cb543ee-dbbe-4059-9c0f-403dc37cf8b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0293bb8ce55146de837bc39fb1a88d3f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:19.834304 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2cb543ee-dbbe-4059-9c0f-403dc37cf8b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0293bb8ce55146de837bc39fb1a88d3f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:19.834910 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2cb543ee-dbbe-4059-9c0f-403dc37cf8b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0293bb8ce55146de837bc39fb1a88d3f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:19.848000 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1325/2628] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:12:19 2022] DELETE /identity/v3/users/0293bb8ce55146de837bc39fb1a88d3f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:19.981141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cf860-c7b7-4ba5-97e0-0a4a355a2815 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:19.981468 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cf860-c7b7-4ba5-97e0-0a4a355a2815 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:19.981711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cf860-c7b7-4ba5-97e0-0a4a355a2815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:19.995741 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:20.006684 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-7c2cf860-c7b7-4ba5-97e0-0a4a355a2815 None None] MFA Rules not processed for user `e98003818ed84c9b8e2301467edebdd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:20.007811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-7c2cf860-c7b7-4ba5-97e0-0a4a355a2815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:20.066947 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1304/2629] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:12:20.136563 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb543ee-dbbe-4059-9c0f-403dc37cf8b3 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:20.142496 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2cb543ee-dbbe-4059-9c0f-403dc37cf8b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:20.158481 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:20.218866 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb543ee-dbbe-4059-9c0f-403dc37cf8b3 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:20.223618 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57773ebc-175e-412b-8b56-b58d6b52eeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:20.224499 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57773ebc-175e-412b-8b56-b58d6b52eeff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:20.224778 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57773ebc-175e-412b-8b56-b58d6b52eeff admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:20.225053 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57773ebc-175e-412b-8b56-b58d6b52eeff admin admin] PATH_INFO: `/v3/projects/11f11d498f7b4a149e363ed5dfc1a28b` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:20.226636 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57773ebc-175e-412b-8b56-b58d6b52eeff admin admin] RBAC: Authorizing `identity:delete_project(project_id=11f11d498f7b4a149e363ed5dfc1a28b)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:20.228592 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57773ebc-175e-412b-8b56-b58d6b52eeff admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:20.284675 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-57773ebc-175e-412b-8b56-b58d6b52eeff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11f11d498f7b4a149e363ed5dfc1a28b', 'request_id': 'req-57773ebc-175e-412b-8b56-b58d6b52eeff'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:20.290963 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-57773ebc-175e-412b-8b56-b58d6b52eeff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11f11d498f7b4a149e363ed5dfc1a28b', 'request_id': 'req-57773ebc-175e-412b-8b56-b58d6b52eeff'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:20.298048 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-57773ebc-175e-412b-8b56-b58d6b52eeff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11f11d498f7b4a149e363ed5dfc1a28b', 'request_id': 'req-57773ebc-175e-412b-8b56-b58d6b52eeff'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:20.312736 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-57773ebc-175e-412b-8b56-b58d6b52eeff admin admin] The token cache is being invalidate because project 11f11d498f7b4a149e363ed5dfc1a28b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:20.313032 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-57773ebc-175e-412b-8b56-b58d6b52eeff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:20.313694 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-57773ebc-175e-412b-8b56-b58d6b52eeff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:20.315408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1326/2630] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:12:20 2022] DELETE /identity/v3/projects/11f11d498f7b4a149e363ed5dfc1a28b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:20.321681 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d37d0f-9ed6-4014-ac73-63a5fdbac44e None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:20.321971 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d37d0f-9ed6-4014-ac73-63a5fdbac44e None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:20.322289 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d37d0f-9ed6-4014-ac73-63a5fdbac44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:20.341588 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-a8d37d0f-9ed6-4014-ac73-63a5fdbac44e None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:20.342902 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-a8d37d0f-9ed6-4014-ac73-63a5fdbac44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:20.420423 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1305/2631] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:12:20.428671 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c31c5-f575-4b16-b125-5f6bd9fdd60b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:20.429579 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c31c5-f575-4b16-b125-5f6bd9fdd60b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:20.429780 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c31c5-f575-4b16-b125-5f6bd9fdd60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:20.452181 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-aa0c31c5-f575-4b16-b125-5f6bd9fdd60b None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:20.453491 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-aa0c31c5-f575-4b16-b125-5f6bd9fdd60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:20.466151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1327/2632] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:20.470735 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d37d0f-9ed6-4014-ac73-63a5fdbac44e None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:20.483039 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d37d0f-9ed6-4014-ac73-63a5fdbac44e None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:20.488449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9a959-532c-4546-90f1-7ae085f8088e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:20.489483 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9a959-532c-4546-90f1-7ae085f8088e admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:20.489776 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9a959-532c-4546-90f1-7ae085f8088e admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:20.490137 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9a959-532c-4546-90f1-7ae085f8088e admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:20.490913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9a959-532c-4546-90f1-7ae085f8088e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:12:20.491785 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9a959-532c-4546-90f1-7ae085f8088e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:20.493502 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9a959-532c-4546-90f1-7ae085f8088e admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:20.508510 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1306/2633] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:12:20.513820 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c31c5-f575-4b16-b125-5f6bd9fdd60b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:20.524906 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c31c5-f575-4b16-b125-5f6bd9fdd60b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:20.534075 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950645ff-b633-4072-be3a-ec8ad2d40b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:20.534793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950645ff-b633-4072-be3a-ec8ad2d40b13 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:20.535079 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950645ff-b633-4072-be3a-ec8ad2d40b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:20.535545 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950645ff-b633-4072-be3a-ec8ad2d40b13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:20.536244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950645ff-b633-4072-be3a-ec8ad2d40b13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:20.538184 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950645ff-b633-4072-be3a-ec8ad2d40b13 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:20.566232 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1328/2634] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:12:20 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:20.570622 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9a959-532c-4546-90f1-7ae085f8088e admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:20.580834 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9a959-532c-4546-90f1-7ae085f8088e admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:20.642382 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120b0bb-94f2-4d24-af66-d8a573bdcfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:20.643047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120b0bb-94f2-4d24-af66-d8a573bdcfae admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:20.643323 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120b0bb-94f2-4d24-af66-d8a573bdcfae admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:20.643602 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120b0bb-94f2-4d24-af66-d8a573bdcfae admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:20.644287 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120b0bb-94f2-4d24-af66-d8a573bdcfae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:20.646227 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120b0bb-94f2-4d24-af66-d8a573bdcfae admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:20.671136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1307/2635] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:12:20 2022] POST /identity/v3/users => generated 284 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:20.675013 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950645ff-b633-4072-be3a-ec8ad2d40b13 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:20.687134 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950645ff-b633-4072-be3a-ec8ad2d40b13 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:20.692383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c77fe7-0537-4614-b34d-0e6c92257884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:20.693199 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c77fe7-0537-4614-b34d-0e6c92257884 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:20.693487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c77fe7-0537-4614-b34d-0e6c92257884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:20.693764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c77fe7-0537-4614-b34d-0e6c92257884 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:20.694632 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c77fe7-0537-4614-b34d-0e6c92257884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:20.696705 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c77fe7-0537-4614-b34d-0e6c92257884 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:20.710245 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1329/2636] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:20.712321 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120b0bb-94f2-4d24-af66-d8a573bdcfae admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:20.732455 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120b0bb-94f2-4d24-af66-d8a573bdcfae admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:20.737569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb08f4-7070-4c05-86b6-22c4499a3751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:20.738496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb08f4-7070-4c05-86b6-22c4499a3751 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:20.738790 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb08f4-7070-4c05-86b6-22c4499a3751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:20.739072 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb08f4-7070-4c05-86b6-22c4499a3751 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:20.739829 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb08f4-7070-4c05-86b6-22c4499a3751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:20.742047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb08f4-7070-4c05-86b6-22c4499a3751 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:20.753327 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1308/2637] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:20.757742 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c77fe7-0537-4614-b34d-0e6c92257884 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:20.768100 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c77fe7-0537-4614-b34d-0e6c92257884 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:20.772875 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19842d3-5714-4f90-831e-a3773c8e2b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:20.774050 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19842d3-5714-4f90-831e-a3773c8e2b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:20.774368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19842d3-5714-4f90-831e-a3773c8e2b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:20.774925 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19842d3-5714-4f90-831e-a3773c8e2b9c admin admin] PATH_INFO: `/v3/projects/3b11dec5e3e24921ba87249341308a67/users/f3dc872c7ff7451b8e5e801877b154ec/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:20.792709 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19842d3-5714-4f90-831e-a3773c8e2b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b11dec5e3e24921ba87249341308a67, user_id=f3dc872c7ff7451b8e5e801877b154ec, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:20.795611 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19842d3-5714-4f90-831e-a3773c8e2b9c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:20.810823 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1330/2638] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:12:20 2022] PUT /identity/v3/projects/3b11dec5e3e24921ba87249341308a67/users/f3dc872c7ff7451b8e5e801877b154ec/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:20.822445 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a19c9f-1950-4529-8640-15813361a144 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:20.822742 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a19c9f-1950-4529-8640-15813361a144 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:20.823025 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a19c9f-1950-4529-8640-15813361a144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:20.852357 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-86a19c9f-1950-4529-8640-15813361a144 None None] MFA Rules not processed for user `f3dc872c7ff7451b8e5e801877b154ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:20.853777 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-86a19c9f-1950-4529-8640-15813361a144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:20.860708 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:20.932928 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1309/2639] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:20 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:20.937918 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19842d3-5714-4f90-831e-a3773c8e2b9c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:20.988700 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:21.018442 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19842d3-5714-4f90-831e-a3773c8e2b9c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:21.025189 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b08666-90a1-4dfe-8ec5-2afd01116e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:21.026169 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b08666-90a1-4dfe-8ec5-2afd01116e5c admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:21.026518 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b08666-90a1-4dfe-8ec5-2afd01116e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:21.026800 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b08666-90a1-4dfe-8ec5-2afd01116e5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:21.027517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b08666-90a1-4dfe-8ec5-2afd01116e5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:21.029838 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b08666-90a1-4dfe-8ec5-2afd01116e5c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:21.059217 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1331/2640] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:12:20 2022] POST /identity/v3/projects => generated 367 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:12:21.063288 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a19c9f-1950-4529-8640-15813361a144 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:21.074769 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a19c9f-1950-4529-8640-15813361a144 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:21.154553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c707d-5db3-43aa-b416-3b7b3524075b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:21.155055 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c707d-5db3-43aa-b416-3b7b3524075b admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:21.155393 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c707d-5db3-43aa-b416-3b7b3524075b admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:21.155613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c707d-5db3-43aa-b416-3b7b3524075b admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:21.157481 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c707d-5db3-43aa-b416-3b7b3524075b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:21.159613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c707d-5db3-43aa-b416-3b7b3524075b admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:21.185580 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1310/2641] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:12:21 2022] POST /identity/v3/users => generated 290 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:21.189794 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b08666-90a1-4dfe-8ec5-2afd01116e5c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:21.201432 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b08666-90a1-4dfe-8ec5-2afd01116e5c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:21.206091 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb7716-efa0-4cbd-a65b-fd6cbaa4f1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:21.206798 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb7716-efa0-4cbd-a65b-fd6cbaa4f1ee admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:21.207090 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb7716-efa0-4cbd-a65b-fd6cbaa4f1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:21.207352 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb7716-efa0-4cbd-a65b-fd6cbaa4f1ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:21.208104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb7716-efa0-4cbd-a65b-fd6cbaa4f1ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:21.209702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb7716-efa0-4cbd-a65b-fd6cbaa4f1ee admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:21.223495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1332/2642] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:21.228956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c707d-5db3-43aa-b416-3b7b3524075b admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:21.240271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c707d-5db3-43aa-b416-3b7b3524075b admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:21.245787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9ec7b-c6e0-4ef3-a1ae-cac81bfec2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:21.246752 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9ec7b-c6e0-4ef3-a1ae-cac81bfec2e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:21.247037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9ec7b-c6e0-4ef3-a1ae-cac81bfec2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:21.247455 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9ec7b-c6e0-4ef3-a1ae-cac81bfec2e3 admin admin] PATH_INFO: `/v3/projects/94cf5c3fe6e24cc39826736d8f8fe77e/users/5f6d5f2b7bb341b2ab0fd485b0ea97b7/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:21.263456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e9ec7b-c6e0-4ef3-a1ae-cac81bfec2e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94cf5c3fe6e24cc39826736d8f8fe77e, user_id=5f6d5f2b7bb341b2ab0fd485b0ea97b7, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:21.266117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e9ec7b-c6e0-4ef3-a1ae-cac81bfec2e3 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:21.282336 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1311/2643] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:12:21 2022] PUT /identity/v3/projects/94cf5c3fe6e24cc39826736d8f8fe77e/users/5f6d5f2b7bb341b2ab0fd485b0ea97b7/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:12:21.286378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb7716-efa0-4cbd-a65b-fd6cbaa4f1ee admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:21.298828 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb7716-efa0-4cbd-a65b-fd6cbaa4f1ee admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:21.369112 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13eaa7-4bb1-4426-9cb9-e8e559a7bbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:21.369793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13eaa7-4bb1-4426-9cb9-e8e559a7bbfc admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:21.370076 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13eaa7-4bb1-4426-9cb9-e8e559a7bbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:21.370395 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13eaa7-4bb1-4426-9cb9-e8e559a7bbfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:21.371118 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a13eaa7-4bb1-4426-9cb9-e8e559a7bbfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:21.372746 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a13eaa7-4bb1-4426-9cb9-e8e559a7bbfc admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:21.383952 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1333/2644] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:21.388512 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9ec7b-c6e0-4ef3-a1ae-cac81bfec2e3 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:21.401049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9ec7b-c6e0-4ef3-a1ae-cac81bfec2e3 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:21.406392 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f02ff-eebc-447c-b6a0-ee59f6dd4929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:21.407320 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f02ff-eebc-447c-b6a0-ee59f6dd4929 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:21.407598 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f02ff-eebc-447c-b6a0-ee59f6dd4929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:21.407871 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f02ff-eebc-447c-b6a0-ee59f6dd4929 admin admin] PATH_INFO: `/v3/projects/94cf5c3fe6e24cc39826736d8f8fe77e/users/5f6d5f2b7bb341b2ab0fd485b0ea97b7/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:21.409867 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f02ff-eebc-447c-b6a0-ee59f6dd4929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94cf5c3fe6e24cc39826736d8f8fe77e, user_id=5f6d5f2b7bb341b2ab0fd485b0ea97b7, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:21.412525 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f02ff-eebc-447c-b6a0-ee59f6dd4929 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:21.423849 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1312/2645] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:12:21 2022] PUT /identity/v3/projects/94cf5c3fe6e24cc39826736d8f8fe77e/users/5f6d5f2b7bb341b2ab0fd485b0ea97b7/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:21.435511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a3a0d-01c2-4974-b4e6-4a718050f056 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:21.435848 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a3a0d-01c2-4974-b4e6-4a718050f056 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:21.436092 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a3a0d-01c2-4974-b4e6-4a718050f056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:21.465664 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5a1a3a0d-01c2-4974-b4e6-4a718050f056 None None] MFA Rules not processed for user `5f6d5f2b7bb341b2ab0fd485b0ea97b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:21.466918 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5a1a3a0d-01c2-4974-b4e6-4a718050f056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:21.536500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1334/2646] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:21 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:21.548632 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f02ff-eebc-447c-b6a0-ee59f6dd4929 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:21.550563 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f77f02ff-eebc-447c-b6a0-ee59f6dd4929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:21.612667 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f02ff-eebc-447c-b6a0-ee59f6dd4929 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:21.616415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7180d7-c056-48c8-ba80-022c9a4b3f09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:21.617308 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7180d7-c056-48c8-ba80-022c9a4b3f09 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:21.617531 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7180d7-c056-48c8-ba80-022c9a4b3f09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:21.617620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7180d7-c056-48c8-ba80-022c9a4b3f09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:21.628579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7180d7-c056-48c8-ba80-022c9a4b3f09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:21.630035 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7180d7-c056-48c8-ba80-022c9a4b3f09 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:21.648907 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1313/2647] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:21 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:21.758385 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a3a0d-01c2-4974-b4e6-4a718050f056 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:21.779252 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a3a0d-01c2-4974-b4e6-4a718050f056 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:21.783625 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3eaae-c69b-40a3-ba73-8d94c2673735 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:21.784297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3eaae-c69b-40a3-ba73-8d94c2673735 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:21.785300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3eaae-c69b-40a3-ba73-8d94c2673735 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:21.785591 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3eaae-c69b-40a3-ba73-8d94c2673735 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:21.826620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:21.867100 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3eaae-c69b-40a3-ba73-8d94c2673735 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:21.869068 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3eaae-c69b-40a3-ba73-8d94c2673735 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:21.878722 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1335/2648] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:21 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:12:22.561662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef4982-d73c-4191-a887-dc0e923fecad None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:22.562076 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef4982-d73c-4191-a887-dc0e923fecad None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:22.562522 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef4982-d73c-4191-a887-dc0e923fecad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:22.568784 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:22.583063 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-ceef4982-d73c-4191-a887-dc0e923fecad None None] MFA Rules not processed for user `3e06d1b0e1a9472db764f4be7e42fdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:22.584438 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-ceef4982-d73c-4191-a887-dc0e923fecad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:22.649795 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1314/2649] 10.222.0.76 () {58 vars in 1009 bytes} [Thu May 26 18:12:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:12:22.701329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3eaae-c69b-40a3-ba73-8d94c2673735 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:22.702800 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f9c3eaae-c69b-40a3-ba73-8d94c2673735 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:22.780167 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3eaae-c69b-40a3-ba73-8d94c2673735 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:22.786773 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4847b-d74c-4342-961a-ec56724f561d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ca774e864ca44cfb187e38bc8a8f6ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:22.787621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4847b-d74c-4342-961a-ec56724f561d service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:22.787750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4847b-d74c-4342-961a-ec56724f561d service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:22.787979 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4847b-d74c-4342-961a-ec56724f561d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:22.798559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4847b-d74c-4342-961a-ec56724f561d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:22.800215 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4847b-d74c-4342-961a-ec56724f561d service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:22.811061 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1336/2650] 10.222.0.76 () {60 vars in 1384 bytes} [Thu May 26 18:12:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:12:27.617536 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef4982-d73c-4191-a887-dc0e923fecad None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:27.739439 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef4982-d73c-4191-a887-dc0e923fecad None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:27.751557 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64e0da-7c27-4240-8927-1b8a56ddc300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:27.753439 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64e0da-7c27-4240-8927-1b8a56ddc300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:27.754131 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64e0da-7c27-4240-8927-1b8a56ddc300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:27.754854 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64e0da-7c27-4240-8927-1b8a56ddc300 admin admin] PATH_INFO: `/v3/users/f3dc872c7ff7451b8e5e801877b154ec` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:27.757496 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64e0da-7c27-4240-8927-1b8a56ddc300 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3dc872c7ff7451b8e5e801877b154ec)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:27.761568 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64e0da-7c27-4240-8927-1b8a56ddc300 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:27.856409 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-4e64e0da-7c27-4240-8927-1b8a56ddc300 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3dc872c7ff7451b8e5e801877b154ec'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:27.871053 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-4e64e0da-7c27-4240-8927-1b8a56ddc300 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3dc872c7ff7451b8e5e801877b154ec'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:27.871738 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-4e64e0da-7c27-4240-8927-1b8a56ddc300 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3dc872c7ff7451b8e5e801877b154ec'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:27.872351 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-4e64e0da-7c27-4240-8927-1b8a56ddc300 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3dc872c7ff7451b8e5e801877b154ec'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:27.883314 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-4e64e0da-7c27-4240-8927-1b8a56ddc300 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3dc872c7ff7451b8e5e801877b154ec'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:27.922316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1315/2651] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:12:27 2022] DELETE /identity/v3/users/f3dc872c7ff7451b8e5e801877b154ec => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:27.927457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099e0f9-eaec-4658-acf6-20fb63d65012 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:27.928007 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099e0f9-eaec-4658-acf6-20fb63d65012 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:27.928007 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099e0f9-eaec-4658-acf6-20fb63d65012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:27.956778 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b099e0f9-eaec-4658-acf6-20fb63d65012 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:27.956778 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b099e0f9-eaec-4658-acf6-20fb63d65012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:28.099228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1337/2652] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:28.112467 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64e0da-7c27-4240-8927-1b8a56ddc300 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:28.117538 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-4e64e0da-7c27-4240-8927-1b8a56ddc300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:28.192780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:28.271170 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64e0da-7c27-4240-8927-1b8a56ddc300 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:28.284436 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62d404-b823-4faf-8c33-882a43186399 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:28.285802 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62d404-b823-4faf-8c33-882a43186399 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:28.286131 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62d404-b823-4faf-8c33-882a43186399 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:28.286485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62d404-b823-4faf-8c33-882a43186399 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:28.302534 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62d404-b823-4faf-8c33-882a43186399 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:28.304928 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62d404-b823-4faf-8c33-882a43186399 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:28.319990 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1316/2653] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:28.753324 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099e0f9-eaec-4658-acf6-20fb63d65012 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:28.754673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b099e0f9-eaec-4658-acf6-20fb63d65012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:28.771281 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099e0f9-eaec-4658-acf6-20fb63d65012 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:28.780276 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3153f0-697a-4561-8505-0634dee8d8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:28.781169 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3153f0-697a-4561-8505-0634dee8d8d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:28.785401 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3153f0-697a-4561-8505-0634dee8d8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:28.785401 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3153f0-697a-4561-8505-0634dee8d8d6 admin admin] PATH_INFO: `/v3/projects/3b11dec5e3e24921ba87249341308a67` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:28.785401 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3153f0-697a-4561-8505-0634dee8d8d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b11dec5e3e24921ba87249341308a67)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:28.786310 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3153f0-697a-4561-8505-0634dee8d8d6 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:28.861821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5e3153f0-697a-4561-8505-0634dee8d8d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b11dec5e3e24921ba87249341308a67', 'request_id': 'req-5e3153f0-697a-4561-8505-0634dee8d8d6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:28.869635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:28.873800 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5e3153f0-697a-4561-8505-0634dee8d8d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b11dec5e3e24921ba87249341308a67', 'request_id': 'req-5e3153f0-697a-4561-8505-0634dee8d8d6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:28.886750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5e3153f0-697a-4561-8505-0634dee8d8d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b11dec5e3e24921ba87249341308a67', 'request_id': 'req-5e3153f0-697a-4561-8505-0634dee8d8d6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:28.907635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5e3153f0-697a-4561-8505-0634dee8d8d6 admin admin] The token cache is being invalidate because project 3b11dec5e3e24921ba87249341308a67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:28.907965 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5e3153f0-697a-4561-8505-0634dee8d8d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:28.908959 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5e3153f0-697a-4561-8505-0634dee8d8d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:28.914292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1338/2654] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:12:28 2022] DELETE /identity/v3/projects/3b11dec5e3e24921ba87249341308a67 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:28.919374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62d404-b823-4faf-8c33-882a43186399 service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:28.935548 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62d404-b823-4faf-8c33-882a43186399 service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:29.013191 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057d705-379c-4e0a-b456-993ae3be48ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:29.014004 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057d705-379c-4e0a-b456-993ae3be48ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:29.014324 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057d705-379c-4e0a-b456-993ae3be48ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:29.014606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057d705-379c-4e0a-b456-993ae3be48ba admin admin] PATH_INFO: `/v3/users/5f6d5f2b7bb341b2ab0fd485b0ea97b7` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:29.015808 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057d705-379c-4e0a-b456-993ae3be48ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f6d5f2b7bb341b2ab0fd485b0ea97b7)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:29.017810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057d705-379c-4e0a-b456-993ae3be48ba admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:29.087620 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-e057d705-379c-4e0a-b456-993ae3be48ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f6d5f2b7bb341b2ab0fd485b0ea97b7'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:29.095683 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-e057d705-379c-4e0a-b456-993ae3be48ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f6d5f2b7bb341b2ab0fd485b0ea97b7'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:29.096394 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-e057d705-379c-4e0a-b456-993ae3be48ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f6d5f2b7bb341b2ab0fd485b0ea97b7'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:29.097012 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-e057d705-379c-4e0a-b456-993ae3be48ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f6d5f2b7bb341b2ab0fd485b0ea97b7'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:29.106374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-e057d705-379c-4e0a-b456-993ae3be48ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f6d5f2b7bb341b2ab0fd485b0ea97b7'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:29.127814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1317/2655] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:12:28 2022] DELETE /identity/v3/users/5f6d5f2b7bb341b2ab0fd485b0ea97b7 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:29.662745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3153f0-697a-4561-8505-0634dee8d8d6 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:29.667724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5e3153f0-697a-4561-8505-0634dee8d8d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:29.696623 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3153f0-697a-4561-8505-0634dee8d8d6 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:29.764536 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced5ed0-56e9-4577-805a-40f86044cadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:29.765587 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced5ed0-56e9-4577-805a-40f86044cadb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:29.766066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced5ed0-56e9-4577-805a-40f86044cadb admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:29.766490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced5ed0-56e9-4577-805a-40f86044cadb admin admin] PATH_INFO: `/v3/projects/94cf5c3fe6e24cc39826736d8f8fe77e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:29.767771 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced5ed0-56e9-4577-805a-40f86044cadb admin admin] RBAC: Authorizing `identity:delete_project(project_id=94cf5c3fe6e24cc39826736d8f8fe77e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:29.769924 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced5ed0-56e9-4577-805a-40f86044cadb admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:29.837788 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9ced5ed0-56e9-4577-805a-40f86044cadb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94cf5c3fe6e24cc39826736d8f8fe77e', 'request_id': 'req-9ced5ed0-56e9-4577-805a-40f86044cadb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:29.843570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9ced5ed0-56e9-4577-805a-40f86044cadb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94cf5c3fe6e24cc39826736d8f8fe77e', 'request_id': 'req-9ced5ed0-56e9-4577-805a-40f86044cadb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:29.852688 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9ced5ed0-56e9-4577-805a-40f86044cadb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94cf5c3fe6e24cc39826736d8f8fe77e', 'request_id': 'req-9ced5ed0-56e9-4577-805a-40f86044cadb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:29.870442 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9ced5ed0-56e9-4577-805a-40f86044cadb admin admin] The token cache is being invalidate because project 94cf5c3fe6e24cc39826736d8f8fe77e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:29.870806 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9ced5ed0-56e9-4577-805a-40f86044cadb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:29.871567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9ced5ed0-56e9-4577-805a-40f86044cadb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:29.873427 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1339/2656] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:12:29 2022] DELETE /identity/v3/projects/94cf5c3fe6e24cc39826736d8f8fe77e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:29.878797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028feb6-8e8e-4ee9-9180-8ed8f7bd68aa None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:29.879165 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028feb6-8e8e-4ee9-9180-8ed8f7bd68aa None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:29.879517 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028feb6-8e8e-4ee9-9180-8ed8f7bd68aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:29.904383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-b028feb6-8e8e-4ee9-9180-8ed8f7bd68aa None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:29.905604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-b028feb6-8e8e-4ee9-9180-8ed8f7bd68aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:29.978228 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:30.007843 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1318/2657] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:12:30.009698 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce1f73-7a50-44c5-b711-2429ff259afb None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:30.010017 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce1f73-7a50-44c5-b711-2429ff259afb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:30.010368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce1f73-7a50-44c5-b711-2429ff259afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:30.032645 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-cbce1f73-7a50-44c5-b711-2429ff259afb None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:30.033887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cbce1f73-7a50-44c5-b711-2429ff259afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:30.043720 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1340/2658] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:30.047852 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028feb6-8e8e-4ee9-9180-8ed8f7bd68aa None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:30.059551 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028feb6-8e8e-4ee9-9180-8ed8f7bd68aa None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:30.065211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffb04b-97f6-49e6-881d-2fe8fdd8a6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:30.066092 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffb04b-97f6-49e6-881d-2fe8fdd8a6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:30.066495 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffb04b-97f6-49e6-881d-2fe8fdd8a6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:30.066855 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffb04b-97f6-49e6-881d-2fe8fdd8a6a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:30.067686 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffb04b-97f6-49e6-881d-2fe8fdd8a6a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:12:30.068031 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffb04b-97f6-49e6-881d-2fe8fdd8a6a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:30.070253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffb04b-97f6-49e6-881d-2fe8fdd8a6a3 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:30.084297 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1319/2659] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:12:30.088972 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce1f73-7a50-44c5-b711-2429ff259afb None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:30.105790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce1f73-7a50-44c5-b711-2429ff259afb None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:30.115239 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e65dd-9300-4a37-8919-b3911544269d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:30.116944 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e65dd-9300-4a37-8919-b3911544269d admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:30.117461 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e65dd-9300-4a37-8919-b3911544269d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:30.118122 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e65dd-9300-4a37-8919-b3911544269d admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:30.119152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e65dd-9300-4a37-8919-b3911544269d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:30.121164 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e65dd-9300-4a37-8919-b3911544269d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:30.157742 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1341/2660] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:12:30 2022] POST /identity/v3/projects => generated 381 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:30.162616 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffb04b-97f6-49e6-881d-2fe8fdd8a6a3 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:30.181223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffb04b-97f6-49e6-881d-2fe8fdd8a6a3 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:30.255162 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcab78f-55f1-4c3f-b77c-de45cafb363c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:30.255905 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcab78f-55f1-4c3f-b77c-de45cafb363c admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:30.256369 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcab78f-55f1-4c3f-b77c-de45cafb363c admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:30.257249 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcab78f-55f1-4c3f-b77c-de45cafb363c admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:30.258008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcab78f-55f1-4c3f-b77c-de45cafb363c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:30.260028 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcab78f-55f1-4c3f-b77c-de45cafb363c admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:30.286502 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1320/2661] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:12:30 2022] POST /identity/v3/users => generated 291 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:30.291262 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e65dd-9300-4a37-8919-b3911544269d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:30.302524 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e65dd-9300-4a37-8919-b3911544269d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:30.311251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d70a7-0106-4335-9d54-9ef2e8515206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:30.311966 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d70a7-0106-4335-9d54-9ef2e8515206 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:30.312236 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d70a7-0106-4335-9d54-9ef2e8515206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:30.312518 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d70a7-0106-4335-9d54-9ef2e8515206 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:30.313256 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d70a7-0106-4335-9d54-9ef2e8515206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:30.315006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d70a7-0106-4335-9d54-9ef2e8515206 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:30.329791 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1342/2662] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:30.333919 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcab78f-55f1-4c3f-b77c-de45cafb363c admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:30.344981 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcab78f-55f1-4c3f-b77c-de45cafb363c admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:30.351590 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec982090-9d45-46be-8d7b-5e0e64da3b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:30.352579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec982090-9d45-46be-8d7b-5e0e64da3b9f admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:30.352866 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec982090-9d45-46be-8d7b-5e0e64da3b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:30.353165 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec982090-9d45-46be-8d7b-5e0e64da3b9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:30.353885 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec982090-9d45-46be-8d7b-5e0e64da3b9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:30.355526 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec982090-9d45-46be-8d7b-5e0e64da3b9f admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:30.372200 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1321/2663] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:12:30.372959 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d70a7-0106-4335-9d54-9ef2e8515206 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:30.385721 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d70a7-0106-4335-9d54-9ef2e8515206 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:30.390834 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87fe95-fe5e-4b9e-83d0-0ca49f1346ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:30.391815 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87fe95-fe5e-4b9e-83d0-0ca49f1346ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:30.392175 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87fe95-fe5e-4b9e-83d0-0ca49f1346ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:30.392523 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87fe95-fe5e-4b9e-83d0-0ca49f1346ef admin admin] PATH_INFO: `/v3/projects/ec4220f75f3e482d85a9a099166ec23b/users/c18ea6245e3346f3834c8855fd1380cb/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:30.412183 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87fe95-fe5e-4b9e-83d0-0ca49f1346ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec4220f75f3e482d85a9a099166ec23b, user_id=c18ea6245e3346f3834c8855fd1380cb, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:30.416716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87fe95-fe5e-4b9e-83d0-0ca49f1346ef admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:30.432804 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1343/2664] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:12:30 2022] PUT /identity/v3/projects/ec4220f75f3e482d85a9a099166ec23b/users/c18ea6245e3346f3834c8855fd1380cb/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:12:30.442865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513e40f-f951-4b9c-a9b4-6815110c1fd8 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:30.443202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513e40f-f951-4b9c-a9b4-6815110c1fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:30.443495 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513e40f-f951-4b9c-a9b4-6815110c1fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:30.480212 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-7513e40f-f951-4b9c-a9b4-6815110c1fd8 None None] MFA Rules not processed for user `c18ea6245e3346f3834c8855fd1380cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:30.481773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-7513e40f-f951-4b9c-a9b4-6815110c1fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:30.567280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1322/2665] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:30.571901 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87fe95-fe5e-4b9e-83d0-0ca49f1346ef admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:30.592074 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:30.690507 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87fe95-fe5e-4b9e-83d0-0ca49f1346ef admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:30.701624 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898b031-1917-40ef-92fb-d4de6f9a3ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:30.702524 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898b031-1917-40ef-92fb-d4de6f9a3ce0 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:30.702907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898b031-1917-40ef-92fb-d4de6f9a3ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:30.703287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898b031-1917-40ef-92fb-d4de6f9a3ce0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:30.704126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0898b031-1917-40ef-92fb-d4de6f9a3ce0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:30.706214 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0898b031-1917-40ef-92fb-d4de6f9a3ce0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:30.747503 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1344/2666] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:12:30 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:30.752686 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513e40f-f951-4b9c-a9b4-6815110c1fd8 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:30.769687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513e40f-f951-4b9c-a9b4-6815110c1fd8 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:30.858159 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:30.882494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d4162-a8eb-4ec6-b671-5ecc2f472734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:30.883264 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d4162-a8eb-4ec6-b671-5ecc2f472734 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:30.883467 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d4162-a8eb-4ec6-b671-5ecc2f472734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:30.883733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d4162-a8eb-4ec6-b671-5ecc2f472734 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:30.884613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31d4162-a8eb-4ec6-b671-5ecc2f472734 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:30.886600 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31d4162-a8eb-4ec6-b671-5ecc2f472734 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:30.914969 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1323/2667] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:12:30 2022] POST /identity/v3/users => generated 296 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:30.918734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898b031-1917-40ef-92fb-d4de6f9a3ce0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:30.931297 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898b031-1917-40ef-92fb-d4de6f9a3ce0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:30.938227 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca89857-c28f-406b-936d-1651f3402ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:30.939070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca89857-c28f-406b-936d-1651f3402ccf admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:30.939458 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca89857-c28f-406b-936d-1651f3402ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:30.939840 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca89857-c28f-406b-936d-1651f3402ccf admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:30.940740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca89857-c28f-406b-936d-1651f3402ccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:30.943586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca89857-c28f-406b-936d-1651f3402ccf admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:30.957072 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1345/2668] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:12:30.962165 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d4162-a8eb-4ec6-b671-5ecc2f472734 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:30.972696 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d4162-a8eb-4ec6-b671-5ecc2f472734 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:30.978158 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f61c3-1f3a-48d7-8608-a84a952be598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:30.979314 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f61c3-1f3a-48d7-8608-a84a952be598 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:30.979974 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f61c3-1f3a-48d7-8608-a84a952be598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:30.979974 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f61c3-1f3a-48d7-8608-a84a952be598 admin admin] PATH_INFO: `/v3/projects/564b096f93b4403c91fe0f7f4f139928/users/feb267b12b914d4488432dc3e6305644/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:31.001589 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f61c3-1f3a-48d7-8608-a84a952be598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=564b096f93b4403c91fe0f7f4f139928, user_id=feb267b12b914d4488432dc3e6305644, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:31.012032 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f61c3-1f3a-48d7-8608-a84a952be598 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:31.040458 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1324/2669] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:12:30 2022] PUT /identity/v3/projects/564b096f93b4403c91fe0f7f4f139928/users/feb267b12b914d4488432dc3e6305644/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:31.044599 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca89857-c28f-406b-936d-1651f3402ccf admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:31.057768 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca89857-c28f-406b-936d-1651f3402ccf admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:31.160606 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e776f7a-6614-479b-ba55-6219851db634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:31.161371 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e776f7a-6614-479b-ba55-6219851db634 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:31.161734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e776f7a-6614-479b-ba55-6219851db634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:31.162087 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e776f7a-6614-479b-ba55-6219851db634 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:31.163519 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e776f7a-6614-479b-ba55-6219851db634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:31.165248 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e776f7a-6614-479b-ba55-6219851db634 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:31.180258 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1346/2670] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:31.182579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f61c3-1f3a-48d7-8608-a84a952be598 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:31.199530 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f61c3-1f3a-48d7-8608-a84a952be598 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:31.205743 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ef4dd-b081-4b12-8c0a-374202f01ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:31.206804 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ef4dd-b081-4b12-8c0a-374202f01ae6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:31.207119 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ef4dd-b081-4b12-8c0a-374202f01ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:31.207427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ef4dd-b081-4b12-8c0a-374202f01ae6 admin admin] PATH_INFO: `/v3/projects/564b096f93b4403c91fe0f7f4f139928/users/feb267b12b914d4488432dc3e6305644/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:31.209571 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ef4dd-b081-4b12-8c0a-374202f01ae6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=564b096f93b4403c91fe0f7f4f139928, user_id=feb267b12b914d4488432dc3e6305644, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:31.212977 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ef4dd-b081-4b12-8c0a-374202f01ae6 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:31.225912 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1325/2671] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:12:31 2022] PUT /identity/v3/projects/564b096f93b4403c91fe0f7f4f139928/users/feb267b12b914d4488432dc3e6305644/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:31.294491 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584801a-3d2d-4634-888a-76746fc1d71d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:31.295164 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584801a-3d2d-4634-888a-76746fc1d71d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:31.295753 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584801a-3d2d-4634-888a-76746fc1d71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:31.338020 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0584801a-3d2d-4634-888a-76746fc1d71d None None] MFA Rules not processed for user `feb267b12b914d4488432dc3e6305644`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:31.339764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0584801a-3d2d-4634-888a-76746fc1d71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:31.463411 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1347/2672] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:31.475137 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ef4dd-b081-4b12-8c0a-374202f01ae6 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:31.477336 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-fa4ef4dd-b081-4b12-8c0a-374202f01ae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:31.574241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ef4dd-b081-4b12-8c0a-374202f01ae6 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:31.578127 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97338c82-6875-48e0-80a2-b113fe61353f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:31.579003 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97338c82-6875-48e0-80a2-b113fe61353f service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:31.579362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97338c82-6875-48e0-80a2-b113fe61353f service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:31.579714 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97338c82-6875-48e0-80a2-b113fe61353f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:31.592206 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97338c82-6875-48e0-80a2-b113fe61353f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:31.593803 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97338c82-6875-48e0-80a2-b113fe61353f service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:31.616841 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1326/2673] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:32.182640 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584801a-3d2d-4634-888a-76746fc1d71d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:32.206759 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:32.324547 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584801a-3d2d-4634-888a-76746fc1d71d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:32.337586 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d51c22-1165-41ac-b92b-cdc8d1fe327a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:32.339599 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d51c22-1165-41ac-b92b-cdc8d1fe327a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:32.340354 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d51c22-1165-41ac-b92b-cdc8d1fe327a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:32.341027 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d51c22-1165-41ac-b92b-cdc8d1fe327a admin admin] PATH_INFO: `/v3/users/c18ea6245e3346f3834c8855fd1380cb` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:32.343560 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d51c22-1165-41ac-b92b-cdc8d1fe327a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c18ea6245e3346f3834c8855fd1380cb)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:32.347095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d51c22-1165-41ac-b92b-cdc8d1fe327a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:32.423455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-63d51c22-1165-41ac-b92b-cdc8d1fe327a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c18ea6245e3346f3834c8855fd1380cb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:32.433419 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-63d51c22-1165-41ac-b92b-cdc8d1fe327a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c18ea6245e3346f3834c8855fd1380cb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:32.454814 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-63d51c22-1165-41ac-b92b-cdc8d1fe327a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c18ea6245e3346f3834c8855fd1380cb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:32.464534 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-63d51c22-1165-41ac-b92b-cdc8d1fe327a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c18ea6245e3346f3834c8855fd1380cb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:32.465373 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-63d51c22-1165-41ac-b92b-cdc8d1fe327a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c18ea6245e3346f3834c8855fd1380cb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:32.485852 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1348/2674] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:12:32 2022] DELETE /identity/v3/users/c18ea6245e3346f3834c8855fd1380cb => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:32.492072 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7edbd-4094-4670-81e1-247c4911b5b8 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:32.492599 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7edbd-4094-4670-81e1-247c4911b5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:32.492697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7edbd-4094-4670-81e1-247c4911b5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:32.525147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-f6a7edbd-4094-4670-81e1-247c4911b5b8 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:32.526591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f6a7edbd-4094-4670-81e1-247c4911b5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:32.638944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1327/2675] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:32.645847 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d51c22-1165-41ac-b92b-cdc8d1fe327a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:32.648859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-63d51c22-1165-41ac-b92b-cdc8d1fe327a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:32.735482 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d51c22-1165-41ac-b92b-cdc8d1fe327a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:32.739846 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5560c0-6fdc-4804-ac22-57e216242d1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:32.740652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5560c0-6fdc-4804-ac22-57e216242d1b service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:32.741104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5560c0-6fdc-4804-ac22-57e216242d1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:32.742242 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5560c0-6fdc-4804-ac22-57e216242d1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:32.758303 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5560c0-6fdc-4804-ac22-57e216242d1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:32.760231 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5560c0-6fdc-4804-ac22-57e216242d1b service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:32.778321 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1349/2676] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:33.190561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7edbd-4094-4670-81e1-247c4911b5b8 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:33.192255 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f6a7edbd-4094-4670-81e1-247c4911b5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:33.207395 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7edbd-4094-4670-81e1-247c4911b5b8 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:33.212886 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a92df-eaea-4457-a8fb-5dcdbcb132a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:33.213588 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a92df-eaea-4457-a8fb-5dcdbcb132a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:33.213977 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a92df-eaea-4457-a8fb-5dcdbcb132a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:33.214389 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a92df-eaea-4457-a8fb-5dcdbcb132a3 admin admin] PATH_INFO: `/v3/projects/ec4220f75f3e482d85a9a099166ec23b` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:33.215809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a92df-eaea-4457-a8fb-5dcdbcb132a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec4220f75f3e482d85a9a099166ec23b)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:33.218317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a92df-eaea-4457-a8fb-5dcdbcb132a3 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:33.270085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:33.297389 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-b05a92df-eaea-4457-a8fb-5dcdbcb132a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec4220f75f3e482d85a9a099166ec23b', 'request_id': 'req-b05a92df-eaea-4457-a8fb-5dcdbcb132a3'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:33.310371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-b05a92df-eaea-4457-a8fb-5dcdbcb132a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec4220f75f3e482d85a9a099166ec23b', 'request_id': 'req-b05a92df-eaea-4457-a8fb-5dcdbcb132a3'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:33.330622 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-b05a92df-eaea-4457-a8fb-5dcdbcb132a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec4220f75f3e482d85a9a099166ec23b', 'request_id': 'req-b05a92df-eaea-4457-a8fb-5dcdbcb132a3'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:33.347552 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-b05a92df-eaea-4457-a8fb-5dcdbcb132a3 admin admin] The token cache is being invalidate because project ec4220f75f3e482d85a9a099166ec23b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:33.348066 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-b05a92df-eaea-4457-a8fb-5dcdbcb132a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:33.348835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-b05a92df-eaea-4457-a8fb-5dcdbcb132a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:33.354542 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1328/2677] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:12:33 2022] DELETE /identity/v3/projects/ec4220f75f3e482d85a9a099166ec23b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:33.354800 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5560c0-6fdc-4804-ac22-57e216242d1b service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:33.369308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5560c0-6fdc-4804-ac22-57e216242d1b service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:33.411597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:33.452555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97ac4b-e76b-4250-8811-dc9f66203ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:33.453696 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97ac4b-e76b-4250-8811-dc9f66203ca6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:33.454068 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97ac4b-e76b-4250-8811-dc9f66203ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:33.454430 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97ac4b-e76b-4250-8811-dc9f66203ca6 admin admin] PATH_INFO: `/v3/users/feb267b12b914d4488432dc3e6305644` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:33.455810 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97ac4b-e76b-4250-8811-dc9f66203ca6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=feb267b12b914d4488432dc3e6305644)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:33.457827 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97ac4b-e76b-4250-8811-dc9f66203ca6 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:33.546750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3a97ac4b-e76b-4250-8811-dc9f66203ca6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feb267b12b914d4488432dc3e6305644'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:33.563149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3a97ac4b-e76b-4250-8811-dc9f66203ca6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feb267b12b914d4488432dc3e6305644'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:33.591159 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3a97ac4b-e76b-4250-8811-dc9f66203ca6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feb267b12b914d4488432dc3e6305644'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:33.592152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3a97ac4b-e76b-4250-8811-dc9f66203ca6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feb267b12b914d4488432dc3e6305644'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:33.592976 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3a97ac4b-e76b-4250-8811-dc9f66203ca6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feb267b12b914d4488432dc3e6305644'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:33.609721 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1350/2678] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:12:33 2022] DELETE /identity/v3/users/feb267b12b914d4488432dc3e6305644 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:34.121491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a92df-eaea-4457-a8fb-5dcdbcb132a3 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:34.124490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-b05a92df-eaea-4457-a8fb-5dcdbcb132a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:34.157003 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a92df-eaea-4457-a8fb-5dcdbcb132a3 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:34.224514 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f317f21b-cb7b-4814-ab0b-ee09306f0d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:34.225809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317f21b-cb7b-4814-ab0b-ee09306f0d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:34.226327 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317f21b-cb7b-4814-ab0b-ee09306f0d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:34.226611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317f21b-cb7b-4814-ab0b-ee09306f0d53 admin admin] PATH_INFO: `/v3/projects/564b096f93b4403c91fe0f7f4f139928` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:34.227839 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f317f21b-cb7b-4814-ab0b-ee09306f0d53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=564b096f93b4403c91fe0f7f4f139928)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:34.229756 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f317f21b-cb7b-4814-ab0b-ee09306f0d53 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:34.278427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f317f21b-cb7b-4814-ab0b-ee09306f0d53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '564b096f93b4403c91fe0f7f4f139928', 'request_id': 'req-f317f21b-cb7b-4814-ab0b-ee09306f0d53'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:34.287087 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f317f21b-cb7b-4814-ab0b-ee09306f0d53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '564b096f93b4403c91fe0f7f4f139928', 'request_id': 'req-f317f21b-cb7b-4814-ab0b-ee09306f0d53'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:34.293273 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f317f21b-cb7b-4814-ab0b-ee09306f0d53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '564b096f93b4403c91fe0f7f4f139928', 'request_id': 'req-f317f21b-cb7b-4814-ab0b-ee09306f0d53'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:34.304904 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f317f21b-cb7b-4814-ab0b-ee09306f0d53 admin admin] The token cache is being invalidate because project 564b096f93b4403c91fe0f7f4f139928 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:34.305186 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f317f21b-cb7b-4814-ab0b-ee09306f0d53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:34.305858 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f317f21b-cb7b-4814-ab0b-ee09306f0d53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:34.307514 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1329/2679] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:12:34 2022] DELETE /identity/v3/projects/564b096f93b4403c91fe0f7f4f139928 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:34.312585 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd788b39-8ca7-46c7-96fa-801a615fdc31 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:34.312877 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd788b39-8ca7-46c7-96fa-801a615fdc31 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:34.313160 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd788b39-8ca7-46c7-96fa-801a615fdc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:34.335295 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-bd788b39-8ca7-46c7-96fa-801a615fdc31 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:34.336559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bd788b39-8ca7-46c7-96fa-801a615fdc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:34.404315 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1351/2680] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:34.412298 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffe24e-e36c-4530-b86e-5403efa2900c None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:34.412609 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffe24e-e36c-4530-b86e-5403efa2900c None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:34.412922 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffe24e-e36c-4530-b86e-5403efa2900c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:34.435782 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-63ffe24e-e36c-4530-b86e-5403efa2900c None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:34.436994 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-63ffe24e-e36c-4530-b86e-5403efa2900c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:34.445812 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1330/2681] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:34.450233 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd788b39-8ca7-46c7-96fa-801a615fdc31 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:34.461883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd788b39-8ca7-46c7-96fa-801a615fdc31 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:34.466558 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb37502-751f-46c4-a2bf-6c9aad8c0255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:34.467246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb37502-751f-46c4-a2bf-6c9aad8c0255 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:34.467528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb37502-751f-46c4-a2bf-6c9aad8c0255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:34.467803 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb37502-751f-46c4-a2bf-6c9aad8c0255 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:34.468545 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb37502-751f-46c4-a2bf-6c9aad8c0255 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:12:34.468858 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb37502-751f-46c4-a2bf-6c9aad8c0255 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:34.470517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb37502-751f-46c4-a2bf-6c9aad8c0255 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:34.484394 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1352/2682] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:12:34.489034 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffe24e-e36c-4530-b86e-5403efa2900c None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:34.499485 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffe24e-e36c-4530-b86e-5403efa2900c None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:34.504593 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922c906-05ce-4541-9d76-8f2316b686c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:34.505464 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922c906-05ce-4541-9d76-8f2316b686c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:34.505640 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922c906-05ce-4541-9d76-8f2316b686c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:34.505912 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922c906-05ce-4541-9d76-8f2316b686c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:34.507947 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922c906-05ce-4541-9d76-8f2316b686c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:34.510004 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922c906-05ce-4541-9d76-8f2316b686c9 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:34.539359 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1331/2683] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:12:34 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:34.543606 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb37502-751f-46c4-a2bf-6c9aad8c0255 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:34.554553 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb37502-751f-46c4-a2bf-6c9aad8c0255 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:34.591945 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:34.621249 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cd31b-9dfa-4291-b1d1-259250382e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:34.622261 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cd31b-9dfa-4291-b1d1-259250382e7a admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:34.622757 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cd31b-9dfa-4291-b1d1-259250382e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:34.622880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cd31b-9dfa-4291-b1d1-259250382e7a admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:34.623752 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0cd31b-9dfa-4291-b1d1-259250382e7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:34.625759 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0cd31b-9dfa-4291-b1d1-259250382e7a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:34.647982 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1353/2684] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:12:34 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:12:34.652086 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922c906-05ce-4541-9d76-8f2316b686c9 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:34.663853 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922c906-05ce-4541-9d76-8f2316b686c9 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:34.669024 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71432d58-7f34-4043-8714-5b5bd30b7b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:34.669752 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71432d58-7f34-4043-8714-5b5bd30b7b57 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:34.670037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71432d58-7f34-4043-8714-5b5bd30b7b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:34.670351 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71432d58-7f34-4043-8714-5b5bd30b7b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:34.671093 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71432d58-7f34-4043-8714-5b5bd30b7b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:34.672764 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71432d58-7f34-4043-8714-5b5bd30b7b57 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:34.686017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1332/2685] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:34.690219 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cd31b-9dfa-4291-b1d1-259250382e7a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:34.701697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cd31b-9dfa-4291-b1d1-259250382e7a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:34.706965 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43914208-46a3-4d6d-9bba-02d6277bd7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:34.707655 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43914208-46a3-4d6d-9bba-02d6277bd7c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:34.707936 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43914208-46a3-4d6d-9bba-02d6277bd7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:34.708274 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43914208-46a3-4d6d-9bba-02d6277bd7c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:34.709011 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43914208-46a3-4d6d-9bba-02d6277bd7c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:34.710898 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43914208-46a3-4d6d-9bba-02d6277bd7c0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:34.722228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1354/2686] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:12:34.727167 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71432d58-7f34-4043-8714-5b5bd30b7b57 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:34.746085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71432d58-7f34-4043-8714-5b5bd30b7b57 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:34.753688 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c0185-9f41-4dd2-bb48-9ab98e608b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:34.754713 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c0185-9f41-4dd2-bb48-9ab98e608b35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:34.754999 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c0185-9f41-4dd2-bb48-9ab98e608b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:34.755317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c0185-9f41-4dd2-bb48-9ab98e608b35 admin admin] PATH_INFO: `/v3/projects/c4d32ade1ba94dde849cc9879465bf41/users/644d40d005ad4fbeadd87bc9d240b122/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:34.772328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:34.798571 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c0185-9f41-4dd2-bb48-9ab98e608b35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4d32ade1ba94dde849cc9879465bf41, user_id=644d40d005ad4fbeadd87bc9d240b122, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:34.800162 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c0185-9f41-4dd2-bb48-9ab98e608b35 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:34.824278 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1333/2687] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:12:34 2022] PUT /identity/v3/projects/c4d32ade1ba94dde849cc9879465bf41/users/644d40d005ad4fbeadd87bc9d240b122/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:12:34.836863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e0e46-0940-4ff7-b087-0c5286f22e9c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:34.837186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e0e46-0940-4ff7-b087-0c5286f22e9c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:34.837458 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e0e46-0940-4ff7-b087-0c5286f22e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:34.869592 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f28e0e46-0940-4ff7-b087-0c5286f22e9c None None] MFA Rules not processed for user `644d40d005ad4fbeadd87bc9d240b122`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:34.871012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f28e0e46-0940-4ff7-b087-0c5286f22e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:34.953747 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1355/2688] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:12:34.964778 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c0185-9f41-4dd2-bb48-9ab98e608b35 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:34.966657 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-460c0185-9f41-4dd2-bb48-9ab98e608b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:35.028956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c0185-9f41-4dd2-bb48-9ab98e608b35 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:35.032622 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3eec1a-4af9-4822-a930-af0232d30849 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:35.033280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3eec1a-4af9-4822-a930-af0232d30849 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:35.033562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3eec1a-4af9-4822-a930-af0232d30849 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:35.033835 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3eec1a-4af9-4822-a930-af0232d30849 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:35.044204 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3eec1a-4af9-4822-a930-af0232d30849 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:35.045698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3eec1a-4af9-4822-a930-af0232d30849 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:35.056219 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1334/2689] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:35.184070 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5edd5-f127-42d8-b534-4249d0746ccd None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:35.184204 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5edd5-f127-42d8-b534-4249d0746ccd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:35.184477 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5edd5-f127-42d8-b534-4249d0746ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:35.207602 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2df5edd5-f127-42d8-b534-4249d0746ccd None None] MFA Rules not processed for user `e98003818ed84c9b8e2301467edebdd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:35.208753 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2df5edd5-f127-42d8-b534-4249d0746ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:35.297332 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1356/2690] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:35.608491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3eec1a-4af9-4822-a930-af0232d30849 service cinder] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:35.610090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-1b3eec1a-4af9-4822-a930-af0232d30849 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:35.683698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3eec1a-4af9-4822-a930-af0232d30849 service cinder] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:35.687424 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785fd07-269e-40a1-9653-a10363c531be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:35.688123 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785fd07-269e-40a1-9653-a10363c531be service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:35.688408 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785fd07-269e-40a1-9653-a10363c531be service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:35.688684 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785fd07-269e-40a1-9653-a10363c531be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:35.690663 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-1785fd07-269e-40a1-9653-a10363c531be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:35.702991 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785fd07-269e-40a1-9653-a10363c531be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:35.704466 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785fd07-269e-40a1-9653-a10363c531be service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:35.714719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1335/2691] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:12:38.324597 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5edd5-f127-42d8-b534-4249d0746ccd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:38.385186 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:38.405294 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5edd5-f127-42d8-b534-4249d0746ccd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:38.411139 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35589ad-e018-43d5-ba4a-54d3fb62974c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:38.412002 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35589ad-e018-43d5-ba4a-54d3fb62974c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:38.412288 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35589ad-e018-43d5-ba4a-54d3fb62974c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:38.412621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35589ad-e018-43d5-ba4a-54d3fb62974c admin admin] PATH_INFO: `/v3/users/644d40d005ad4fbeadd87bc9d240b122` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:38.413889 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35589ad-e018-43d5-ba4a-54d3fb62974c admin admin] RBAC: Authorizing `identity:delete_user(user_id=644d40d005ad4fbeadd87bc9d240b122)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:38.415902 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35589ad-e018-43d5-ba4a-54d3fb62974c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:38.472106 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f35589ad-e018-43d5-ba4a-54d3fb62974c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '644d40d005ad4fbeadd87bc9d240b122'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:38.482008 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f35589ad-e018-43d5-ba4a-54d3fb62974c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '644d40d005ad4fbeadd87bc9d240b122'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:38.496244 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f35589ad-e018-43d5-ba4a-54d3fb62974c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '644d40d005ad4fbeadd87bc9d240b122'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:38.497117 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f35589ad-e018-43d5-ba4a-54d3fb62974c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '644d40d005ad4fbeadd87bc9d240b122'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:38.497817 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f35589ad-e018-43d5-ba4a-54d3fb62974c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '644d40d005ad4fbeadd87bc9d240b122'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:38.512106 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1357/2692] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:12:38 2022] DELETE /identity/v3/users/644d40d005ad4fbeadd87bc9d240b122 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:38.516616 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3bda0-6cdb-4e8d-b2fc-6c1d80ea7e16 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:38.517096 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3bda0-6cdb-4e8d-b2fc-6c1d80ea7e16 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:38.517246 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3bda0-6cdb-4e8d-b2fc-6c1d80ea7e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:38.538726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-4fb3bda0-6cdb-4e8d-b2fc-6c1d80ea7e16 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:38.540420 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-4fb3bda0-6cdb-4e8d-b2fc-6c1d80ea7e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:38.613721 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1336/2693] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:12:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:38.624776 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35589ad-e018-43d5-ba4a-54d3fb62974c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:38.626337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f35589ad-e018-43d5-ba4a-54d3fb62974c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:38.699308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35589ad-e018-43d5-ba4a-54d3fb62974c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:38.705621 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0e530-15eb-4e24-9ae8-90d12cef141e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:38.707238 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0e530-15eb-4e24-9ae8-90d12cef141e service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:38.707687 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0e530-15eb-4e24-9ae8-90d12cef141e service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:38.708336 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0e530-15eb-4e24-9ae8-90d12cef141e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:38.721275 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0e530-15eb-4e24-9ae8-90d12cef141e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:38.723081 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0e530-15eb-4e24-9ae8-90d12cef141e service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:38.737843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1358/2694] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:12:39.095170 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3bda0-6cdb-4e8d-b2fc-6c1d80ea7e16 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:39.097023 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-4fb3bda0-6cdb-4e8d-b2fc-6c1d80ea7e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:39.118927 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3bda0-6cdb-4e8d-b2fc-6c1d80ea7e16 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:39.151911 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5153a-6372-42db-83f0-744ba768e60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:39.153281 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5153a-6372-42db-83f0-744ba768e60d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:39.153722 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5153a-6372-42db-83f0-744ba768e60d admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:39.154048 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5153a-6372-42db-83f0-744ba768e60d admin admin] PATH_INFO: `/v3/projects/c4d32ade1ba94dde849cc9879465bf41` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:39.156131 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c5153a-6372-42db-83f0-744ba768e60d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4d32ade1ba94dde849cc9879465bf41)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:39.159527 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c5153a-6372-42db-83f0-744ba768e60d admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:39.201769 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:39.227606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-20c5153a-6372-42db-83f0-744ba768e60d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4d32ade1ba94dde849cc9879465bf41', 'request_id': 'req-20c5153a-6372-42db-83f0-744ba768e60d'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:39.239853 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-20c5153a-6372-42db-83f0-744ba768e60d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4d32ade1ba94dde849cc9879465bf41', 'request_id': 'req-20c5153a-6372-42db-83f0-744ba768e60d'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:39.252918 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-20c5153a-6372-42db-83f0-744ba768e60d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4d32ade1ba94dde849cc9879465bf41', 'request_id': 'req-20c5153a-6372-42db-83f0-744ba768e60d'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:39.280971 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-20c5153a-6372-42db-83f0-744ba768e60d admin admin] The token cache is being invalidate because project c4d32ade1ba94dde849cc9879465bf41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:12:39.280971 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-20c5153a-6372-42db-83f0-744ba768e60d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:39.280971 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-20c5153a-6372-42db-83f0-744ba768e60d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:12:39.282262 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1337/2695] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:12:39 2022] DELETE /identity/v3/projects/c4d32ade1ba94dde849cc9879465bf41 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:12:39.420097 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217398a-bd35-4395-9f87-92c6486db475 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:39.420932 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217398a-bd35-4395-9f87-92c6486db475 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:39.421280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217398a-bd35-4395-9f87-92c6486db475 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:39.425221 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1359/2696] 10.222.0.76 () {54 vars in 921 bytes} [Thu May 26 18:12:39 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 18:12:39.441583 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac0c9e-901f-44a2-92a6-ae84b4dc1da3 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:39.442134 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac0c9e-901f-44a2-92a6-ae84b4dc1da3 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:39.442748 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac0c9e-901f-44a2-92a6-ae84b4dc1da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:39.492170 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-53ac0c9e-901f-44a2-92a6-ae84b4dc1da3 None None] MFA Rules not processed for user `1899c317083d40f5b490cd9eb63fe521`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:39.496985 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-53ac0c9e-901f-44a2-92a6-ae84b4dc1da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:39.644200 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1338/2697] 10.222.0.76 () {58 vars in 1018 bytes} [Thu May 26 18:12:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:12:39.661304 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217398a-bd35-4395-9f87-92c6486db475 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:39.663305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7217398a-bd35-4395-9f87-92c6486db475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:39.758931 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217398a-bd35-4395-9f87-92c6486db475 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:39.765782 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b714b-5b46-497a-be1d-25bb4c27aa0e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f4c9fc949c4a2c951b755234848cc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:39.767035 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b714b-5b46-497a-be1d-25bb4c27aa0e service glance] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:39.767467 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b714b-5b46-497a-be1d-25bb4c27aa0e service glance] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:39.767970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b714b-5b46-497a-be1d-25bb4c27aa0e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:39.787263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b714b-5b46-497a-be1d-25bb4c27aa0e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:39.788801 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b714b-5b46-497a-be1d-25bb4c27aa0e service glance] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:39.801116 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1360/2698] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:39.884628 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f6282-b227-450d-a1d0-a4cf5528fe61 None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:39.884913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f6282-b227-450d-a1d0-a4cf5528fe61 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:39.885384 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f6282-b227-450d-a1d0-a4cf5528fe61 None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:39.887094 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1339/2699] 10.222.0.76 () {54 vars in 921 bytes} [Thu May 26 18:12:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:12:39.892819 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f78a3d-c3be-4914-8291-2a3bb15a7302 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:39.893801 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f78a3d-c3be-4914-8291-2a3bb15a7302 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:39.894094 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f78a3d-c3be-4914-8291-2a3bb15a7302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:39.915637 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-27f78a3d-c3be-4914-8291-2a3bb15a7302 None None] MFA Rules not processed for user `8df66990a3734209a350c8b6768d259f`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:39.917210 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-27f78a3d-c3be-4914-8291-2a3bb15a7302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:39.927921 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:40.016947 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1361/2700] 10.222.0.76 () {58 vars in 1018 bytes} [Thu May 26 18:12:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:40.031249 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f6282-b227-450d-a1d0-a4cf5528fe61 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:40.033052 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-386f6282-b227-450d-a1d0-a4cf5528fe61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:40.048743 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f6282-b227-450d-a1d0-a4cf5528fe61 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:40.053890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b502acc-506c-43fd-b099-290ade9c4770 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:40.054580 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b502acc-506c-43fd-b099-290ade9c4770 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:40.054859 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b502acc-506c-43fd-b099-290ade9c4770 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:40.055127 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b502acc-506c-43fd-b099-290ade9c4770 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:40.064142 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b502acc-506c-43fd-b099-290ade9c4770 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:40.065605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b502acc-506c-43fd-b099-290ade9c4770 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:40.075513 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1340/2701] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:12:40.175104 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d950db6-c787-434b-a51f-f6156b71fadc None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:40.175756 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d950db6-c787-434b-a51f-f6156b71fadc None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:40.176341 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d950db6-c787-434b-a51f-f6156b71fadc None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:40.178856 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1362/2702] 10.222.0.76 () {54 vars in 921 bytes} [Thu May 26 18:12:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:12:40.193868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdadcfa-c0bb-4737-89fc-69d3e0b51aaf None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:40.194604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdadcfa-c0bb-4737-89fc-69d3e0b51aaf None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:40.194970 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdadcfa-c0bb-4737-89fc-69d3e0b51aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:40.215924 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-4bdadcfa-c0bb-4737-89fc-69d3e0b51aaf None None] MFA Rules not processed for user `1899c317083d40f5b490cd9eb63fe521`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:40.217940 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-4bdadcfa-c0bb-4737-89fc-69d3e0b51aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:40.228491 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1341/2703] 10.222.0.76 () {58 vars in 1018 bytes} [Thu May 26 18:12:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:40.241684 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d950db6-c787-434b-a51f-f6156b71fadc None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:40.253367 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d950db6-c787-434b-a51f-f6156b71fadc None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:40.257346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20cbd1-65fc-4372-8613-a0b3ab03d843 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:40.257990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20cbd1-65fc-4372-8613-a0b3ab03d843 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:40.258309 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20cbd1-65fc-4372-8613-a0b3ab03d843 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:40.258599 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20cbd1-65fc-4372-8613-a0b3ab03d843 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:40.270235 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f20cbd1-65fc-4372-8613-a0b3ab03d843 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:40.271778 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f20cbd1-65fc-4372-8613-a0b3ab03d843 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:40.284926 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1363/2704] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:43.571380 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbee4b-fb4c-4118-86fd-75242ed0c6ae None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:43.572018 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbee4b-fb4c-4118-86fd-75242ed0c6ae None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:43.572018 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbee4b-fb4c-4118-86fd-75242ed0c6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:43.592809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-2dbbee4b-fb4c-4118-86fd-75242ed0c6ae None None] MFA Rules not processed for user `3e06d1b0e1a9472db764f4be7e42fdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:12:43.594134 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-2dbbee4b-fb4c-4118-86fd-75242ed0c6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:43.657798 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1342/2705] 10.222.0.76 () {58 vars in 1009 bytes} [Thu May 26 18:12:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:12:43.671005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20cbd1-65fc-4372-8613-a0b3ab03d843 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:43.672418 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8f20cbd1-65fc-4372-8613-a0b3ab03d843 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:43.744222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20cbd1-65fc-4372-8613-a0b3ab03d843 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:43.749229 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b436f39-8dd4-4045-8a78-f06123c1a9b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ca774e864ca44cfb187e38bc8a8f6ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:43.749883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b436f39-8dd4-4045-8a78-f06123c1a9b0 service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:43.750170 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b436f39-8dd4-4045-8a78-f06123c1a9b0 service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:43.750481 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b436f39-8dd4-4045-8a78-f06123c1a9b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:43.758987 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b436f39-8dd4-4045-8a78-f06123c1a9b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:43.760583 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b436f39-8dd4-4045-8a78-f06123c1a9b0 service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:43.768880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1364/2706] 10.222.0.76 () {60 vars in 1384 bytes} [Thu May 26 18:12:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:12:53.650836 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=78,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:53.765571 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=55 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:12:55.514101 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbbee4b-fb4c-4118-86fd-75242ed0c6ae None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:55.516020 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-2dbbee4b-fb4c-4118-86fd-75242ed0c6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:55.593610 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbbee4b-fb4c-4118-86fd-75242ed0c6ae None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:55.598747 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c85c3-461c-4945-bd47-f5559cdf8135 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '596f126382e3440fb3cf7c357eef889f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:55.599433 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c85c3-461c-4945-bd47-f5559cdf8135 service placement] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:55.599723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c85c3-461c-4945-bd47-f5559cdf8135 service placement] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:55.600005 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c85c3-461c-4945-bd47-f5559cdf8135 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:55.602639 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-c48c85c3-461c-4945-bd47-f5559cdf8135 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:55.614572 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c85c3-461c-4945-bd47-f5559cdf8135 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:55.616207 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c85c3-461c-4945-bd47-f5559cdf8135 service placement] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:55.626560 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1343/2707] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:55.685249 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b436f39-8dd4-4045-8a78-f06123c1a9b0 service swift] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:55.686728 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8b436f39-8dd4-4045-8a78-f06123c1a9b0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:55.700388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b436f39-8dd4-4045-8a78-f06123c1a9b0 service swift] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:55.704149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8271f9a-9f61-4b14-b25a-3cbbee354d91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:55.704800 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8271f9a-9f61-4b14-b25a-3cbbee354d91 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:55.705085 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8271f9a-9f61-4b14-b25a-3cbbee354d91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:55.705362 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8271f9a-9f61-4b14-b25a-3cbbee354d91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:55.707235 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d8271f9a-9f61-4b14-b25a-3cbbee354d91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:55.769838 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8271f9a-9f61-4b14-b25a-3cbbee354d91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:55.771511 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8271f9a-9f61-4b14-b25a-3cbbee354d91 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:55.781794 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1365/2708] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:12:58.779231 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c85c3-461c-4945-bd47-f5559cdf8135 service placement] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:12:58.780640 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-c48c85c3-461c-4945-bd47-f5559cdf8135 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:58.839873 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c85c3-461c-4945-bd47-f5559cdf8135 service placement] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:12:58.843596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961e0fa-358f-42db-ab15-889c8373a6c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:12:58.844280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961e0fa-358f-42db-ab15-889c8373a6c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:12:58.844568 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961e0fa-358f-42db-ab15-889c8373a6c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:12:58.844849 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961e0fa-358f-42db-ab15-889c8373a6c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:12:58.846696 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-2961e0fa-358f-42db-ab15-889c8373a6c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:12:58.917153 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2961e0fa-358f-42db-ab15-889c8373a6c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:12:58.918656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2961e0fa-358f-42db-ab15-889c8373a6c2 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:12:58.928451 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1344/2709] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:12:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:13:05.403460 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8271f9a-9f61-4b14-b25a-3cbbee354d91 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:05.415313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8271f9a-9f61-4b14-b25a-3cbbee354d91 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:05.416850 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e006633-f7a0-4dcf-8394-9c74fbcc0bc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:05.417491 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e006633-f7a0-4dcf-8394-9c74fbcc0bc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:05.417778 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e006633-f7a0-4dcf-8394-9c74fbcc0bc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:05.418056 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e006633-f7a0-4dcf-8394-9c74fbcc0bc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:05.419919 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9e006633-f7a0-4dcf-8394-9c74fbcc0bc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:05.484540 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e006633-f7a0-4dcf-8394-9c74fbcc0bc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:05.486138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e006633-f7a0-4dcf-8394-9c74fbcc0bc8 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:05.496057 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1366/2710] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:13:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:13:05.841675 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985e819-fabf-481e-a882-97c42858b05c None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:05.842171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985e819-fabf-481e-a882-97c42858b05c None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:05.842307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985e819-fabf-481e-a882-97c42858b05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:05.860973 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-d985e819-fabf-481e-a882-97c42858b05c None None] MFA Rules not processed for user `42c96a85400941818b658cca8cb56fec`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:13:05.862211 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-d985e819-fabf-481e-a882-97c42858b05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:05.871449 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1345/2711] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:13:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:13:10.499050 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015ec6a-b73b-471b-9199-915d644567eb None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:10.500352 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015ec6a-b73b-471b-9199-915d644567eb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:10.500867 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015ec6a-b73b-471b-9199-915d644567eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:10.524208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0015ec6a-b73b-471b-9199-915d644567eb None None] MFA Rules not processed for user `3e06d1b0e1a9472db764f4be7e42fdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:13:10.525490 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0015ec6a-b73b-471b-9199-915d644567eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:10.533867 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1367/2712] 10.222.0.76 () {58 vars in 1009 bytes} [Thu May 26 18:13:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:13:10.548079 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d985e819-fabf-481e-a882-97c42858b05c None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:10.565262 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d985e819-fabf-481e-a882-97c42858b05c None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:10.571571 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7d832-3bef-4310-bc41-ba1c085ec39b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ca774e864ca44cfb187e38bc8a8f6ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:10.572350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7d832-3bef-4310-bc41-ba1c085ec39b service swift] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:10.572733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7d832-3bef-4310-bc41-ba1c085ec39b service swift] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:10.573040 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7d832-3bef-4310-bc41-ba1c085ec39b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:10.583645 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7d832-3bef-4310-bc41-ba1c085ec39b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:10.585766 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7d832-3bef-4310-bc41-ba1c085ec39b service swift] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:10.595368 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1346/2713] 10.222.0.76 () {60 vars in 1384 bytes} [Thu May 26 18:13:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:13:11.413952 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e348567-2ab1-4fec-a157-d5d1feb468c1 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:11.414253 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e348567-2ab1-4fec-a157-d5d1feb468c1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:11.414563 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e348567-2ab1-4fec-a157-d5d1feb468c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:11.465349 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8e348567-2ab1-4fec-a157-d5d1feb468c1 None None] MFA Rules not processed for user `3e06d1b0e1a9472db764f4be7e42fdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:13:11.467577 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8e348567-2ab1-4fec-a157-d5d1feb468c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:11.496451 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1368/2714] 10.222.0.76 () {58 vars in 967 bytes} [Thu May 26 18:13:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:13:11.520272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7d832-3bef-4310-bc41-ba1c085ec39b service swift] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:11.539935 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7d832-3bef-4310-bc41-ba1c085ec39b service swift] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:11.565909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6080d-74cf-44d7-8908-b94a446167a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ca774e864ca44cfb187e38bc8a8f6ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:11.566754 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6080d-74cf-44d7-8908-b94a446167a7 service swift] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:11.567143 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6080d-74cf-44d7-8908-b94a446167a7 service swift] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:11.567520 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6080d-74cf-44d7-8908-b94a446167a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:11.591154 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e6080d-74cf-44d7-8908-b94a446167a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:11.593244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e6080d-74cf-44d7-8908-b94a446167a7 service swift] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:11.617866 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1347/2715] 10.222.0.76 () {60 vars in 1384 bytes} [Thu May 26 18:13:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:13:14.528988 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e348567-2ab1-4fec-a157-d5d1feb468c1 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:14.530875 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8e348567-2ab1-4fec-a157-d5d1feb468c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:14.567201 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e348567-2ab1-4fec-a157-d5d1feb468c1 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:14.573255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e16dc3e-30a1-4a02-b305-ced1337fc79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:14.574499 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16dc3e-30a1-4a02-b305-ced1337fc79d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:14.575409 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16dc3e-30a1-4a02-b305-ced1337fc79d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:14.575903 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16dc3e-30a1-4a02-b305-ced1337fc79d admin admin] PATH_INFO: `/v3/users/b2ba168683a4433f8a763cb4c250c9a0` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:14.578740 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e16dc3e-30a1-4a02-b305-ced1337fc79d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2ba168683a4433f8a763cb4c250c9a0)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:14.581131 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e16dc3e-30a1-4a02-b305-ced1337fc79d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:14.699654 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9e16dc3e-30a1-4a02-b305-ced1337fc79d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2ba168683a4433f8a763cb4c250c9a0'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:14.718990 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9e16dc3e-30a1-4a02-b305-ced1337fc79d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2ba168683a4433f8a763cb4c250c9a0'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:14.752635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9e16dc3e-30a1-4a02-b305-ced1337fc79d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2ba168683a4433f8a763cb4c250c9a0'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:14.753665 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9e16dc3e-30a1-4a02-b305-ced1337fc79d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2ba168683a4433f8a763cb4c250c9a0'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:14.754733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9e16dc3e-30a1-4a02-b305-ced1337fc79d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2ba168683a4433f8a763cb4c250c9a0'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:14.757382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:13:14.798861 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1369/2716] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:13:14 2022] DELETE /identity/v3/users/b2ba168683a4433f8a763cb4c250c9a0 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:13:15.071396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6080d-74cf-44d7-8908-b94a446167a7 service swift] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:15.085457 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6080d-74cf-44d7-8908-b94a446167a7 service swift] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:15.097829 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:13:15.164432 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef19fc-de32-487f-bd1d-44917451a058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:15.165440 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef19fc-de32-487f-bd1d-44917451a058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:15.165852 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef19fc-de32-487f-bd1d-44917451a058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:15.166230 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef19fc-de32-487f-bd1d-44917451a058 admin admin] PATH_INFO: `/v3/projects/c2cf8fb59b4f4b12900d336f1d91b1eb` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:15.167678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef19fc-de32-487f-bd1d-44917451a058 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2cf8fb59b4f4b12900d336f1d91b1eb)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:15.169824 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef19fc-de32-487f-bd1d-44917451a058 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:15.226065 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-89ef19fc-de32-487f-bd1d-44917451a058 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2cf8fb59b4f4b12900d336f1d91b1eb', 'request_id': 'req-89ef19fc-de32-487f-bd1d-44917451a058'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:15.237203 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-89ef19fc-de32-487f-bd1d-44917451a058 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2cf8fb59b4f4b12900d336f1d91b1eb', 'request_id': 'req-89ef19fc-de32-487f-bd1d-44917451a058'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:15.247246 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-89ef19fc-de32-487f-bd1d-44917451a058 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2cf8fb59b4f4b12900d336f1d91b1eb', 'request_id': 'req-89ef19fc-de32-487f-bd1d-44917451a058'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:15.262323 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-89ef19fc-de32-487f-bd1d-44917451a058 admin admin] The token cache is being invalidate because project c2cf8fb59b4f4b12900d336f1d91b1eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:13:15.262970 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-89ef19fc-de32-487f-bd1d-44917451a058 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:15.263800 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-89ef19fc-de32-487f-bd1d-44917451a058 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:15.266072 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1348/2717] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:13:15 2022] DELETE /identity/v3/projects/c2cf8fb59b4f4b12900d336f1d91b1eb => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:13:15.270145 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e16dc3e-30a1-4a02-b305-ced1337fc79d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:15.272379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9e16dc3e-30a1-4a02-b305-ced1337fc79d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:15.289023 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e16dc3e-30a1-4a02-b305-ced1337fc79d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:15.384379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48367c-c7a5-4824-ab5b-bf805f73a388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:15.385268 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48367c-c7a5-4824-ab5b-bf805f73a388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:15.385697 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48367c-c7a5-4824-ab5b-bf805f73a388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:15.385980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48367c-c7a5-4824-ab5b-bf805f73a388 admin admin] PATH_INFO: `/v3/users/b6ea4a1f33eb490bb5aa9aeea78e3c94` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:15.387425 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48367c-c7a5-4824-ab5b-bf805f73a388 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6ea4a1f33eb490bb5aa9aeea78e3c94)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:15.389424 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48367c-c7a5-4824-ab5b-bf805f73a388 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:15.457287 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bf48367c-c7a5-4824-ab5b-bf805f73a388 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6ea4a1f33eb490bb5aa9aeea78e3c94'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:15.468661 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bf48367c-c7a5-4824-ab5b-bf805f73a388 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6ea4a1f33eb490bb5aa9aeea78e3c94'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:15.486835 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bf48367c-c7a5-4824-ab5b-bf805f73a388 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6ea4a1f33eb490bb5aa9aeea78e3c94'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:15.487829 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bf48367c-c7a5-4824-ab5b-bf805f73a388 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6ea4a1f33eb490bb5aa9aeea78e3c94'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:15.488891 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bf48367c-c7a5-4824-ab5b-bf805f73a388 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6ea4a1f33eb490bb5aa9aeea78e3c94'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:15.504604 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1370/2718] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:13:15 2022] DELETE /identity/v3/users/b6ea4a1f33eb490bb5aa9aeea78e3c94 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:13:15.807972 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef19fc-de32-487f-bd1d-44917451a058 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:15.811172 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-89ef19fc-de32-487f-bd1d-44917451a058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:15.836659 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef19fc-de32-487f-bd1d-44917451a058 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:15.945579 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a798071-34be-43da-a8dd-47b9050198ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:15.947078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a798071-34be-43da-a8dd-47b9050198ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:15.947078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a798071-34be-43da-a8dd-47b9050198ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:15.947398 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a798071-34be-43da-a8dd-47b9050198ca admin admin] PATH_INFO: `/v3/projects/8351057258f14080aa9614468eee2daf` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:15.948815 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a798071-34be-43da-a8dd-47b9050198ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=8351057258f14080aa9614468eee2daf)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:15.950881 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a798071-34be-43da-a8dd-47b9050198ca admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:16.005592 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:13:16.012327 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-7a798071-34be-43da-a8dd-47b9050198ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8351057258f14080aa9614468eee2daf', 'request_id': 'req-7a798071-34be-43da-a8dd-47b9050198ca'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:16.021785 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-7a798071-34be-43da-a8dd-47b9050198ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8351057258f14080aa9614468eee2daf', 'request_id': 'req-7a798071-34be-43da-a8dd-47b9050198ca'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:16.029013 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-7a798071-34be-43da-a8dd-47b9050198ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8351057258f14080aa9614468eee2daf', 'request_id': 'req-7a798071-34be-43da-a8dd-47b9050198ca'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:16.042313 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-7a798071-34be-43da-a8dd-47b9050198ca admin admin] The token cache is being invalidate because project 8351057258f14080aa9614468eee2daf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:13:16.042605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-7a798071-34be-43da-a8dd-47b9050198ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:16.045155 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-7a798071-34be-43da-a8dd-47b9050198ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:16.046268 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1349/2719] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:13:15 2022] DELETE /identity/v3/projects/8351057258f14080aa9614468eee2daf => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:13:25.500278 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=57,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:13:26.039017 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=8,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:13:27.208516 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48367c-c7a5-4824-ab5b-bf805f73a388 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:27.211519 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bf48367c-c7a5-4824-ab5b-bf805f73a388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:27.350101 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48367c-c7a5-4824-ab5b-bf805f73a388 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:27.442340 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a073b-5a43-41a7-a300-c87b79212366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:27.443566 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a073b-5a43-41a7-a300-c87b79212366 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:27.443992 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a073b-5a43-41a7-a300-c87b79212366 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:27.444391 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a073b-5a43-41a7-a300-c87b79212366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:27.447106 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-de2a073b-5a43-41a7-a300-c87b79212366 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:27.570339 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a073b-5a43-41a7-a300-c87b79212366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:27.570339 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a073b-5a43-41a7-a300-c87b79212366 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:27.593713 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1371/2720] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:13:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:13:29.346659 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c52a4-afc3-49bc-9dad-28926e0a857b None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:29.347182 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c52a4-afc3-49bc-9dad-28926e0a857b None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:29.347360 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c52a4-afc3-49bc-9dad-28926e0a857b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:29.367714 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-022c52a4-afc3-49bc-9dad-28926e0a857b None None] MFA Rules not processed for user `3e06d1b0e1a9472db764f4be7e42fdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:13:29.369089 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-022c52a4-afc3-49bc-9dad-28926e0a857b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:29.433809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1350/2721] 10.222.0.76 () {58 vars in 967 bytes} [Thu May 26 18:13:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:13:29.448435 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a073b-5a43-41a7-a300-c87b79212366 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:29.450065 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-de2a073b-5a43-41a7-a300-c87b79212366 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:29.531973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a073b-5a43-41a7-a300-c87b79212366 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:29.546124 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4c09e-2b66-40c9-905b-7cebc99127e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ca774e864ca44cfb187e38bc8a8f6ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:29.546904 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4c09e-2b66-40c9-905b-7cebc99127e0 service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:29.547192 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4c09e-2b66-40c9-905b-7cebc99127e0 service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:29.547471 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4c09e-2b66-40c9-905b-7cebc99127e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:29.556886 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:13:29.560675 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4c09e-2b66-40c9-905b-7cebc99127e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:29.563073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4c09e-2b66-40c9-905b-7cebc99127e0 service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:29.573878 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1372/2722] 10.222.0.76 () {60 vars in 1384 bytes} [Thu May 26 18:13:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:13:31.657317 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c52a4-afc3-49bc-9dad-28926e0a857b None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:31.658811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-022c52a4-afc3-49bc-9dad-28926e0a857b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:31.730223 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c52a4-afc3-49bc-9dad-28926e0a857b None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:31.735063 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af5cd6-eaf2-4156-b8f0-ae26faeedf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:31.735967 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af5cd6-eaf2-4156-b8f0-ae26faeedf13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:31.736249 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af5cd6-eaf2-4156-b8f0-ae26faeedf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:31.736524 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af5cd6-eaf2-4156-b8f0-ae26faeedf13 admin admin] PATH_INFO: `/v3/users/e98003818ed84c9b8e2301467edebdd6` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:31.737838 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af5cd6-eaf2-4156-b8f0-ae26faeedf13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e98003818ed84c9b8e2301467edebdd6)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:31.740006 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af5cd6-eaf2-4156-b8f0-ae26faeedf13 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:31.792009 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-20af5cd6-eaf2-4156-b8f0-ae26faeedf13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e98003818ed84c9b8e2301467edebdd6'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:31.799296 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-20af5cd6-eaf2-4156-b8f0-ae26faeedf13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e98003818ed84c9b8e2301467edebdd6'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:31.799913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-20af5cd6-eaf2-4156-b8f0-ae26faeedf13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e98003818ed84c9b8e2301467edebdd6'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:31.800541 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-20af5cd6-eaf2-4156-b8f0-ae26faeedf13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e98003818ed84c9b8e2301467edebdd6'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:31.808762 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-20af5cd6-eaf2-4156-b8f0-ae26faeedf13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e98003818ed84c9b8e2301467edebdd6'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:31.828391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1351/2723] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:13:31 2022] DELETE /identity/v3/users/e98003818ed84c9b8e2301467edebdd6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:13:32.024182 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4c09e-2b66-40c9-905b-7cebc99127e0 service swift] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:32.043182 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4c09e-2b66-40c9-905b-7cebc99127e0 service swift] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:32.116028 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fdd6b-c720-4d1b-abba-74d5494af7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:32.117296 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fdd6b-c720-4d1b-abba-74d5494af7c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:32.117714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fdd6b-c720-4d1b-abba-74d5494af7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:32.118019 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fdd6b-c720-4d1b-abba-74d5494af7c1 admin admin] PATH_INFO: `/v3/projects/6217142c97d54164a23b27dafacd083f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:32.119509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fdd6b-c720-4d1b-abba-74d5494af7c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6217142c97d54164a23b27dafacd083f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:32.121558 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fdd6b-c720-4d1b-abba-74d5494af7c1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:32.173986 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-897fdd6b-c720-4d1b-abba-74d5494af7c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6217142c97d54164a23b27dafacd083f', 'request_id': 'req-897fdd6b-c720-4d1b-abba-74d5494af7c1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:32.180891 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-897fdd6b-c720-4d1b-abba-74d5494af7c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6217142c97d54164a23b27dafacd083f', 'request_id': 'req-897fdd6b-c720-4d1b-abba-74d5494af7c1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:32.187334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-897fdd6b-c720-4d1b-abba-74d5494af7c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6217142c97d54164a23b27dafacd083f', 'request_id': 'req-897fdd6b-c720-4d1b-abba-74d5494af7c1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:32.201189 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-897fdd6b-c720-4d1b-abba-74d5494af7c1 admin admin] The token cache is being invalidate because project 6217142c97d54164a23b27dafacd083f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:13:32.201485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-897fdd6b-c720-4d1b-abba-74d5494af7c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:32.202149 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-897fdd6b-c720-4d1b-abba-74d5494af7c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:13:32.204026 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1373/2724] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:13:32 2022] DELETE /identity/v3/projects/6217142c97d54164a23b27dafacd083f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:13:32.209660 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acb780-9f51-4547-81ed-bde3d44e4a4a None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:32.209957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acb780-9f51-4547-81ed-bde3d44e4a4a None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:32.210242 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acb780-9f51-4547-81ed-bde3d44e4a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:32.232166 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-65acb780-9f51-4547-81ed-bde3d44e4a4a None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:13:32.233464 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-65acb780-9f51-4547-81ed-bde3d44e4a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:32.253669 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:13:32.303736 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1352/2725] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:13:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:13:32.311710 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccd019-46c5-4135-aea1-2157bbf8d555 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:32.312001 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccd019-46c5-4135-aea1-2157bbf8d555 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:32.312292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccd019-46c5-4135-aea1-2157bbf8d555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:32.334265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-73ccd019-46c5-4135-aea1-2157bbf8d555 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:13:32.335531 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-73ccd019-46c5-4135-aea1-2157bbf8d555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:32.343997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1374/2726] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:13:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:13:32.348516 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acb780-9f51-4547-81ed-bde3d44e4a4a None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:32.360026 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acb780-9f51-4547-81ed-bde3d44e4a4a None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:32.365768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f557b-9db9-4f69-b523-30cbc4000751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:32.366565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f557b-9db9-4f69-b523-30cbc4000751 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:32.366849 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f557b-9db9-4f69-b523-30cbc4000751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:32.367155 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f557b-9db9-4f69-b523-30cbc4000751 admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:32.367951 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0f557b-9db9-4f69-b523-30cbc4000751 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:13:32.367951 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0f557b-9db9-4f69-b523-30cbc4000751 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:32.369934 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0f557b-9db9-4f69-b523-30cbc4000751 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:32.383778 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1353/2727] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:13:32.389219 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ccd019-46c5-4135-aea1-2157bbf8d555 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:32.400319 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ccd019-46c5-4135-aea1-2157bbf8d555 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:32.405733 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45b4fe-040c-4ad4-b981-50f2d3c9fcec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:32.406446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45b4fe-040c-4ad4-b981-50f2d3c9fcec admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:32.406729 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45b4fe-040c-4ad4-b981-50f2d3c9fcec admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:32.407006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45b4fe-040c-4ad4-b981-50f2d3c9fcec admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:32.407709 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45b4fe-040c-4ad4-b981-50f2d3c9fcec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:32.409659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45b4fe-040c-4ad4-b981-50f2d3c9fcec admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:32.440482 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1375/2728] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:13:32 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:13:32.445140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f557b-9db9-4f69-b523-30cbc4000751 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:32.459510 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f557b-9db9-4f69-b523-30cbc4000751 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:32.524083 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5dbe5b-2e0c-4a64-892a-68f2ca86df94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:32.524785 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5dbe5b-2e0c-4a64-892a-68f2ca86df94 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:32.525071 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5dbe5b-2e0c-4a64-892a-68f2ca86df94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:32.525353 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5dbe5b-2e0c-4a64-892a-68f2ca86df94 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:32.526059 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5dbe5b-2e0c-4a64-892a-68f2ca86df94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:32.528347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5dbe5b-2e0c-4a64-892a-68f2ca86df94 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:32.550616 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1354/2729] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:13:32 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:13:32.556228 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45b4fe-040c-4ad4-b981-50f2d3c9fcec admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:32.567198 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45b4fe-040c-4ad4-b981-50f2d3c9fcec admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:32.572222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80117714-85cf-4414-8add-1e941d69f3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:32.572920 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80117714-85cf-4414-8add-1e941d69f3d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:32.573214 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80117714-85cf-4414-8add-1e941d69f3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:32.573488 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80117714-85cf-4414-8add-1e941d69f3d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:32.574265 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80117714-85cf-4414-8add-1e941d69f3d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:32.575952 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80117714-85cf-4414-8add-1e941d69f3d2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:32.588556 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1376/2730] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:13:32.592728 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5dbe5b-2e0c-4a64-892a-68f2ca86df94 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:32.603637 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5dbe5b-2e0c-4a64-892a-68f2ca86df94 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:32.608372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ba96d-25ef-43d5-a874-cb8d67d7a072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:32.609040 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ba96d-25ef-43d5-a874-cb8d67d7a072 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:32.609322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ba96d-25ef-43d5-a874-cb8d67d7a072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:32.609598 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ba96d-25ef-43d5-a874-cb8d67d7a072 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:32.610383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ba96d-25ef-43d5-a874-cb8d67d7a072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:32.611976 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ba96d-25ef-43d5-a874-cb8d67d7a072 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:32.623756 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1355/2731] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:13:32.627958 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80117714-85cf-4414-8add-1e941d69f3d2 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:32.638382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80117714-85cf-4414-8add-1e941d69f3d2 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:32.643362 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a7feb-8c18-466b-8ff4-51049ebbb454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:32.644256 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a7feb-8c18-466b-8ff4-51049ebbb454 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:32.644542 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a7feb-8c18-466b-8ff4-51049ebbb454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:32.644829 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a7feb-8c18-466b-8ff4-51049ebbb454 admin admin] PATH_INFO: `/v3/projects/fd9dfeec33544d148b87d188b4208835/users/a461e083aa424e1dbf1acc0b68b4b1df/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:32.662151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a7feb-8c18-466b-8ff4-51049ebbb454 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd9dfeec33544d148b87d188b4208835, user_id=a461e083aa424e1dbf1acc0b68b4b1df, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:32.665593 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a7feb-8c18-466b-8ff4-51049ebbb454 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:32.679438 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1377/2732] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:13:32 2022] PUT /identity/v3/projects/fd9dfeec33544d148b87d188b4208835/users/a461e083aa424e1dbf1acc0b68b4b1df/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:13:32.684874 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c613172-b383-4b48-ad0c-338326d2729a None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:32.685606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c613172-b383-4b48-ad0c-338326d2729a None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:32.685606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c613172-b383-4b48-ad0c-338326d2729a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:32.703693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-6c613172-b383-4b48-ad0c-338326d2729a None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:13:32.704918 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-6c613172-b383-4b48-ad0c-338326d2729a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:32.766272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1356/2733] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:13:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:13:32.776722 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a7feb-8c18-466b-8ff4-51049ebbb454 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:32.779869 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3a9a7feb-8c18-466b-8ff4-51049ebbb454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:32.806018 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:13:32.844766 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a7feb-8c18-466b-8ff4-51049ebbb454 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:32.848498 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb2f82-c31c-40c0-a8ae-82afd261253b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:32.849769 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb2f82-c31c-40c0-a8ae-82afd261253b service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:32.850056 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb2f82-c31c-40c0-a8ae-82afd261253b service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:32.850395 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb2f82-c31c-40c0-a8ae-82afd261253b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:32.860446 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb2f82-c31c-40c0-a8ae-82afd261253b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:32.861925 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb2f82-c31c-40c0-a8ae-82afd261253b service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:32.872737 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1378/2734] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:13:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:13:33.495942 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5c54b-a391-4926-b4cf-a1602cbefc2f None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:33.496241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5c54b-a391-4926-b4cf-a1602cbefc2f None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:33.496529 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5c54b-a391-4926-b4cf-a1602cbefc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:33.527250 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-03b5c54b-a391-4926-b4cf-a1602cbefc2f None None] MFA Rules not processed for user `a461e083aa424e1dbf1acc0b68b4b1df`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:13:33.528477 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-03b5c54b-a391-4926-b4cf-a1602cbefc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:33.557131 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:13:33.607180 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1357/2735] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:13:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:13:33.619793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb2f82-c31c-40c0-a8ae-82afd261253b service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:13:33.621264 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-edfb2f82-c31c-40c0-a8ae-82afd261253b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:33.687433 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb2f82-c31c-40c0-a8ae-82afd261253b service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:13:33.691218 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03778b27-adfa-4b3c-88a6-6003749446f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:13:33.691882 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03778b27-adfa-4b3c-88a6-6003749446f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:33.692166 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03778b27-adfa-4b3c-88a6-6003749446f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:33.692441 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03778b27-adfa-4b3c-88a6-6003749446f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:33.702546 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03778b27-adfa-4b3c-88a6-6003749446f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:13:33.703986 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03778b27-adfa-4b3c-88a6-6003749446f8 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:13:33.714534 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1379/2736] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:13:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:13:35.068585 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f92d6-bba2-4671-8f30-157bf7322bcb None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:35.069148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f92d6-bba2-4671-8f30-157bf7322bcb None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:35.069148 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f92d6-bba2-4671-8f30-157bf7322bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:35.089108 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-764f92d6-bba2-4671-8f30-157bf7322bcb None None] MFA Rules not processed for user `a461e083aa424e1dbf1acc0b68b4b1df`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:13:35.090410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-764f92d6-bba2-4671-8f30-157bf7322bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:35.097348 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1358/2737] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:13:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:13:43.710350 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=42 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:13:45.081581 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=21 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:13:57.362310 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073177b-b939-4b2f-a5aa-db77142a5921 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:13:57.362857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073177b-b939-4b2f-a5aa-db77142a5921 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:13:57.362857 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073177b-b939-4b2f-a5aa-db77142a5921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:13:57.380961 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b073177b-b939-4b2f-a5aa-db77142a5921 None None] MFA Rules not processed for user `1b262e67242c4384ba5bf0356a08679e`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:13:57.382174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b073177b-b939-4b2f-a5aa-db77142a5921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:13:57.437771 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1380/2738] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:13:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:14:07.431271 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=25 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:14:40.129326 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60164d-9993-49a1-a94a-ecee5add8c4f None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:14:40.129867 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60164d-9993-49a1-a94a-ecee5add8c4f None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:14:40.130037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60164d-9993-49a1-a94a-ecee5add8c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:14:40.150080 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-0e60164d-9993-49a1-a94a-ecee5add8c4f None None] MFA Rules not processed for user `a461e083aa424e1dbf1acc0b68b4b1df`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:14:40.151592 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-0e60164d-9993-49a1-a94a-ecee5add8c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:14:40.158543 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1359/2739] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:14:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:14:46.774915 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073177b-b939-4b2f-a5aa-db77142a5921 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:14:46.786554 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073177b-b939-4b2f-a5aa-db77142a5921 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:14:46.790147 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0288b-cf44-4049-9c27-e43cf6e9ebf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:14:46.790836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0288b-cf44-4049-9c27-e43cf6e9ebf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:14:46.791123 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0288b-cf44-4049-9c27-e43cf6e9ebf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:14:46.791401 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0288b-cf44-4049-9c27-e43cf6e9ebf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:14:46.793184 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-dfb0288b-cf44-4049-9c27-e43cf6e9ebf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:14:46.869321 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0288b-cf44-4049-9c27-e43cf6e9ebf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:14:46.870820 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0288b-cf44-4049-9c27-e43cf6e9ebf2 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:14:46.883205 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1381/2740] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:14:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:14:48.066811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60164d-9993-49a1-a94a-ecee5add8c4f None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:14:48.068254 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-0e60164d-9993-49a1-a94a-ecee5add8c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:14:48.080098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60164d-9993-49a1-a94a-ecee5add8c4f None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:14:48.085047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f42651-0862-4325-8996-6fed4c97700a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:14:48.085722 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f42651-0862-4325-8996-6fed4c97700a service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:14:48.085978 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f42651-0862-4325-8996-6fed4c97700a service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:14:48.086285 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f42651-0862-4325-8996-6fed4c97700a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:14:48.088054 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-73f42651-0862-4325-8996-6fed4c97700a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:14:48.102888 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f42651-0862-4325-8996-6fed4c97700a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:14:48.104339 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f42651-0862-4325-8996-6fed4c97700a service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:14:48.116482 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1360/2741] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:14:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:14:56.877101 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=34 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:14:58.110617 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=9 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:15:15.483125 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0288b-cf44-4049-9c27-e43cf6e9ebf2 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:15.484755 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-dfb0288b-cf44-4049-9c27-e43cf6e9ebf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:15.502663 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0288b-cf44-4049-9c27-e43cf6e9ebf2 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:15.508328 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a4f5b-7283-46f8-9347-691a39fb000f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:15.508995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a4f5b-7283-46f8-9347-691a39fb000f service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:15.509292 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a4f5b-7283-46f8-9347-691a39fb000f service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:15.509559 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a4f5b-7283-46f8-9347-691a39fb000f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:15.511379 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3b7a4f5b-7283-46f8-9347-691a39fb000f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:15.525842 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7a4f5b-7283-46f8-9347-691a39fb000f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:15.527463 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7a4f5b-7283-46f8-9347-691a39fb000f service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:15.539393 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1382/2742] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:15:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:15:17.629162 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a11473-24ae-48cf-beb0-e9fb77a16380 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:17.629761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a11473-24ae-48cf-beb0-e9fb77a16380 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:17.629761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a11473-24ae-48cf-beb0-e9fb77a16380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:17.648074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-76a11473-24ae-48cf-beb0-e9fb77a16380 None None] MFA Rules not processed for user `3e06d1b0e1a9472db764f4be7e42fdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:15:17.649333 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-76a11473-24ae-48cf-beb0-e9fb77a16380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:17.707251 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1361/2743] 10.222.0.76 () {58 vars in 1009 bytes} [Thu May 26 18:15:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:15:17.753502 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a4f5b-7283-46f8-9347-691a39fb000f service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:17.755137 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3b7a4f5b-7283-46f8-9347-691a39fb000f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:17.825867 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a4f5b-7283-46f8-9347-691a39fb000f service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:17.832172 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c79083-4aff-40ca-ae20-3eeda0fa4f7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ca774e864ca44cfb187e38bc8a8f6ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:17.832837 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c79083-4aff-40ca-ae20-3eeda0fa4f7f service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:17.833121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c79083-4aff-40ca-ae20-3eeda0fa4f7f service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:17.833402 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c79083-4aff-40ca-ae20-3eeda0fa4f7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:17.842259 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c79083-4aff-40ca-ae20-3eeda0fa4f7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:17.844080 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c79083-4aff-40ca-ae20-3eeda0fa4f7f service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:17.853882 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1383/2744] 10.222.0.76 () {60 vars in 1384 bytes} [Thu May 26 18:15:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:15:22.221654 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a11473-24ae-48cf-beb0-e9fb77a16380 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:22.235505 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a11473-24ae-48cf-beb0-e9fb77a16380 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:22.246398 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887368ad-b704-4e00-8a44-4fd925b8d91d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:22.247202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887368ad-b704-4e00-8a44-4fd925b8d91d service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:22.247850 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887368ad-b704-4e00-8a44-4fd925b8d91d service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:22.248300 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887368ad-b704-4e00-8a44-4fd925b8d91d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:22.250787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-887368ad-b704-4e00-8a44-4fd925b8d91d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:22.276329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887368ad-b704-4e00-8a44-4fd925b8d91d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:22.279371 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887368ad-b704-4e00-8a44-4fd925b8d91d service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:22.306638 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1362/2745] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:15:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:15:27.850484 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=38 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:15:32.292323 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=31 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:15:33.283344 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c79083-4aff-40ca-ae20-3eeda0fa4f7f service swift] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:33.301405 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c79083-4aff-40ca-ae20-3eeda0fa4f7f service swift] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:33.308081 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac00f94-4375-483c-8c7c-7f7e51eb5306 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:33.309895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac00f94-4375-483c-8c7c-7f7e51eb5306 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:33.309895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac00f94-4375-483c-8c7c-7f7e51eb5306 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:33.309895 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac00f94-4375-483c-8c7c-7f7e51eb5306 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:33.311117 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cac00f94-4375-483c-8c7c-7f7e51eb5306 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:33.329165 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac00f94-4375-483c-8c7c-7f7e51eb5306 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:33.331937 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac00f94-4375-483c-8c7c-7f7e51eb5306 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:33.347712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1384/2746] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:15:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:15:35.153392 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb7ae4-b74a-42c4-8207-3c68de1f011c None None] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:35.154507 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb7ae4-b74a-42c4-8207-3c68de1f011c None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:35.154948 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb7ae4-b74a-42c4-8207-3c68de1f011c None None] PATH_INFO: `/` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:35.156510 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1363/2747] 10.222.0.76 () {54 vars in 921 bytes} [Thu May 26 18:15:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 18:15:35.162978 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d3311-1059-460d-9279-7e97c7d077b7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:35.163513 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d3311-1059-460d-9279-7e97c7d077b7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:35.163749 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d3311-1059-460d-9279-7e97c7d077b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:35.183569 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-126d3311-1059-460d-9279-7e97c7d077b7 None None] MFA Rules not processed for user `1899c317083d40f5b490cd9eb63fe521`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:15:35.184921 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-126d3311-1059-460d-9279-7e97c7d077b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:35.195512 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1385/2748] 10.222.0.76 () {58 vars in 1018 bytes} [Thu May 26 18:15:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:15:35.211033 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb7ae4-b74a-42c4-8207-3c68de1f011c None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:35.212895 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-39eb7ae4-b74a-42c4-8207-3c68de1f011c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:35.275876 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb7ae4-b74a-42c4-8207-3c68de1f011c None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:35.279480 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b4666f-e66e-4de9-a365-8726ec8f9f14 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f4c9fc949c4a2c951b755234848cc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:35.280139 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4666f-e66e-4de9-a365-8726ec8f9f14 service glance] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:35.280434 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4666f-e66e-4de9-a365-8726ec8f9f14 service glance] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:35.280708 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4666f-e66e-4de9-a365-8726ec8f9f14 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:35.292259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b4666f-e66e-4de9-a365-8726ec8f9f14 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:35.293711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b4666f-e66e-4de9-a365-8726ec8f9f14 service glance] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:35.305242 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1364/2749] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:15:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:15:35.508383 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d3311-1059-460d-9279-7e97c7d077b7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:35.509822 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-126d3311-1059-460d-9279-7e97c7d077b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:35.588009 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d3311-1059-460d-9279-7e97c7d077b7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:35.593049 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5ccd3-8806-4b98-8cfc-b4d915725722 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '596f126382e3440fb3cf7c357eef889f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:35.593701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5ccd3-8806-4b98-8cfc-b4d915725722 service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:35.593986 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5ccd3-8806-4b98-8cfc-b4d915725722 service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:35.594295 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5ccd3-8806-4b98-8cfc-b4d915725722 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:35.596724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-40e5ccd3-8806-4b98-8cfc-b4d915725722 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:35.609642 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5ccd3-8806-4b98-8cfc-b4d915725722 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:35.611342 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5ccd3-8806-4b98-8cfc-b4d915725722 service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:35.621775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1386/2750] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:15:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:15:37.288969 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e7734-e783-48d6-b4a4-4bb810552292 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:37.289503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e7734-e783-48d6-b4a4-4bb810552292 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:37.289503 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e7734-e783-48d6-b4a4-4bb810552292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:37.308293 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-636e7734-e783-48d6-b4a4-4bb810552292 None None] MFA Rules not processed for user `3e06d1b0e1a9472db764f4be7e42fdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:15:37.309632 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-636e7734-e783-48d6-b4a4-4bb810552292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:37.317879 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1365/2751] 10.222.0.76 () {58 vars in 1009 bytes} [Thu May 26 18:15:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:15:37.330773 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5ccd3-8806-4b98-8cfc-b4d915725722 service placement] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:37.342493 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5ccd3-8806-4b98-8cfc-b4d915725722 service placement] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:37.348259 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0bbf06-a144-4c14-b22d-5bbbe638b824 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ca774e864ca44cfb187e38bc8a8f6ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:37.348906 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bbf06-a144-4c14-b22d-5bbbe638b824 service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:37.349188 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bbf06-a144-4c14-b22d-5bbbe638b824 service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:37.349472 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bbf06-a144-4c14-b22d-5bbbe638b824 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:37.358097 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0bbf06-a144-4c14-b22d-5bbbe638b824 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:37.360174 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0bbf06-a144-4c14-b22d-5bbbe638b824 service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:37.369671 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1387/2752] 10.222.0.76 () {60 vars in 1384 bytes} [Thu May 26 18:15:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:15:39.131157 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e7734-e783-48d6-b4a4-4bb810552292 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:39.141753 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e7734-e783-48d6-b4a4-4bb810552292 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:39.146739 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501306f-6a0b-4e12-b81f-b3db0bb2e2db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:39.147651 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501306f-6a0b-4e12-b81f-b3db0bb2e2db service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:39.148269 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501306f-6a0b-4e12-b81f-b3db0bb2e2db service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:39.149394 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501306f-6a0b-4e12-b81f-b3db0bb2e2db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:39.151790 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-1501306f-6a0b-4e12-b81f-b3db0bb2e2db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:39.167561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501306f-6a0b-4e12-b81f-b3db0bb2e2db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:39.169074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501306f-6a0b-4e12-b81f-b3db0bb2e2db service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:39.187684 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1366/2753] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:15:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:15:41.208898 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0bbf06-a144-4c14-b22d-5bbbe638b824 service swift] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:41.211347 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bd0bbf06-a144-4c14-b22d-5bbbe638b824 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:41.230573 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0bbf06-a144-4c14-b22d-5bbbe638b824 service swift] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:41.236826 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0930f-83e2-4935-a1dc-d0218230d9ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '596f126382e3440fb3cf7c357eef889f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:41.237799 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0930f-83e2-4935-a1dc-d0218230d9ba service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:41.238355 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0930f-83e2-4935-a1dc-d0218230d9ba service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:41.238828 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0930f-83e2-4935-a1dc-d0218230d9ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:41.240898 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-75d0930f-83e2-4935-a1dc-d0218230d9ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:41.256489 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d0930f-83e2-4935-a1dc-d0218230d9ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:41.258533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d0930f-83e2-4935-a1dc-d0218230d9ba service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:41.271282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1388/2754] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:15:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:15:41.328837 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501306f-6a0b-4e12-b81f-b3db0bb2e2db service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:41.344659 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501306f-6a0b-4e12-b81f-b3db0bb2e2db service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:41.365875 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a46fbf-8a80-4ec6-b062-8d04d8641f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:41.366920 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a46fbf-8a80-4ec6-b062-8d04d8641f6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:41.367272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a46fbf-8a80-4ec6-b062-8d04d8641f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:41.367624 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a46fbf-8a80-4ec6-b062-8d04d8641f6f admin admin] PATH_INFO: `/v3/users/a461e083aa424e1dbf1acc0b68b4b1df` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:41.369023 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a46fbf-8a80-4ec6-b062-8d04d8641f6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a461e083aa424e1dbf1acc0b68b4b1df)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:41.371085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a46fbf-8a80-4ec6-b062-8d04d8641f6f admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:41.477880 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f6a46fbf-8a80-4ec6-b062-8d04d8641f6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a461e083aa424e1dbf1acc0b68b4b1df'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:41.496339 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f6a46fbf-8a80-4ec6-b062-8d04d8641f6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a461e083aa424e1dbf1acc0b68b4b1df'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:41.498136 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f6a46fbf-8a80-4ec6-b062-8d04d8641f6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a461e083aa424e1dbf1acc0b68b4b1df'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:41.499518 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f6a46fbf-8a80-4ec6-b062-8d04d8641f6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a461e083aa424e1dbf1acc0b68b4b1df'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:41.514363 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-f6a46fbf-8a80-4ec6-b062-8d04d8641f6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a461e083aa424e1dbf1acc0b68b4b1df'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:41.578284 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1367/2755] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:15:41 2022] DELETE /identity/v3/users/a461e083aa424e1dbf1acc0b68b4b1df => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:15:41.895538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0930f-83e2-4935-a1dc-d0218230d9ba service placement] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:41.910368 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0930f-83e2-4935-a1dc-d0218230d9ba service placement] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:42.020738 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae0b2c-297d-41dd-a4b8-d5adc91b188d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:42.023707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae0b2c-297d-41dd-a4b8-d5adc91b188d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:42.024066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae0b2c-297d-41dd-a4b8-d5adc91b188d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:42.025261 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae0b2c-297d-41dd-a4b8-d5adc91b188d admin admin] PATH_INFO: `/v3/projects/fd9dfeec33544d148b87d188b4208835` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:42.043472 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae0b2c-297d-41dd-a4b8-d5adc91b188d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd9dfeec33544d148b87d188b4208835)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:42.046338 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae0b2c-297d-41dd-a4b8-d5adc91b188d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:42.147259 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:15:42.150854 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f9ae0b2c-297d-41dd-a4b8-d5adc91b188d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd9dfeec33544d148b87d188b4208835', 'request_id': 'req-f9ae0b2c-297d-41dd-a4b8-d5adc91b188d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:42.164885 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f9ae0b2c-297d-41dd-a4b8-d5adc91b188d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd9dfeec33544d148b87d188b4208835', 'request_id': 'req-f9ae0b2c-297d-41dd-a4b8-d5adc91b188d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:42.180601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f9ae0b2c-297d-41dd-a4b8-d5adc91b188d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd9dfeec33544d148b87d188b4208835', 'request_id': 'req-f9ae0b2c-297d-41dd-a4b8-d5adc91b188d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:42.199965 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f9ae0b2c-297d-41dd-a4b8-d5adc91b188d admin admin] The token cache is being invalidate because project fd9dfeec33544d148b87d188b4208835 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:15:42.200382 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f9ae0b2c-297d-41dd-a4b8-d5adc91b188d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:42.201248 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f9ae0b2c-297d-41dd-a4b8-d5adc91b188d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:42.208230 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1389/2756] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:15:41 2022] DELETE /identity/v3/projects/fd9dfeec33544d148b87d188b4208835 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:15:42.210091 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542bc15-e420-41cc-94a2-7954a3d7af0a None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:42.210417 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542bc15-e420-41cc-94a2-7954a3d7af0a None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:42.210704 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542bc15-e420-41cc-94a2-7954a3d7af0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:42.267723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-3542bc15-e420-41cc-94a2-7954a3d7af0a None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:15:42.268538 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-3542bc15-e420-41cc-94a2-7954a3d7af0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:42.386074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:15:42.416965 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1368/2757] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:15:42.424907 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b28c5-0eb8-4888-9d97-708b11fe0fbd None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:42.425500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b28c5-0eb8-4888-9d97-708b11fe0fbd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:42.425500 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b28c5-0eb8-4888-9d97-708b11fe0fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:42.453695 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5b3b28c5-0eb8-4888-9d97-708b11fe0fbd None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:15:42.456375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5b3b28c5-0eb8-4888-9d97-708b11fe0fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:42.465718 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1390/2758] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:15:42.469989 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542bc15-e420-41cc-94a2-7954a3d7af0a None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:42.484459 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542bc15-e420-41cc-94a2-7954a3d7af0a None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:42.490229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96902cbf-1265-4033-bfe4-2768a91c3720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:42.490930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96902cbf-1265-4033-bfe4-2768a91c3720 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:42.491229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96902cbf-1265-4033-bfe4-2768a91c3720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:42.491553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96902cbf-1265-4033-bfe4-2768a91c3720 admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:42.492351 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96902cbf-1265-4033-bfe4-2768a91c3720 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:15:42.492668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96902cbf-1265-4033-bfe4-2768a91c3720 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:42.494398 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96902cbf-1265-4033-bfe4-2768a91c3720 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:42.512730 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1369/2759] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:15:42.518234 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b28c5-0eb8-4888-9d97-708b11fe0fbd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:42.535152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b28c5-0eb8-4888-9d97-708b11fe0fbd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:42.542306 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559a781-7934-4ecd-bc5d-c0593988b967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:42.543095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559a781-7934-4ecd-bc5d-c0593988b967 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:42.543387 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559a781-7934-4ecd-bc5d-c0593988b967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:42.543673 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559a781-7934-4ecd-bc5d-c0593988b967 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:42.544580 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5559a781-7934-4ecd-bc5d-c0593988b967 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:42.547386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5559a781-7934-4ecd-bc5d-c0593988b967 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:42.613615 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1391/2760] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:15:42 2022] POST /identity/v3/projects => generated 399 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:15:42.614913 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96902cbf-1265-4033-bfe4-2768a91c3720 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:42.644900 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96902cbf-1265-4033-bfe4-2768a91c3720 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:42.785725 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a76078-a8f9-4b4b-9fed-a66edcc6c70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:42.787137 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a76078-a8f9-4b4b-9fed-a66edcc6c70e admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:42.787678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a76078-a8f9-4b4b-9fed-a66edcc6c70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:42.788183 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a76078-a8f9-4b4b-9fed-a66edcc6c70e admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:42.789116 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a76078-a8f9-4b4b-9fed-a66edcc6c70e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:42.791354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a76078-a8f9-4b4b-9fed-a66edcc6c70e admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:42.840438 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1370/2761] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:15:42 2022] POST /identity/v3/users => generated 300 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:15:42.845966 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559a781-7934-4ecd-bc5d-c0593988b967 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:42.865181 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559a781-7934-4ecd-bc5d-c0593988b967 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:42.873780 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a90c3d-697f-4c61-9a04-c0c4fba99713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:42.874659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a90c3d-697f-4c61-9a04-c0c4fba99713 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:42.875043 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a90c3d-697f-4c61-9a04-c0c4fba99713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:42.875425 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a90c3d-697f-4c61-9a04-c0c4fba99713 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:42.877487 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a90c3d-697f-4c61-9a04-c0c4fba99713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:42.879334 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a90c3d-697f-4c61-9a04-c0c4fba99713 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:42.926875 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1392/2762] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:15:42.930954 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a76078-a8f9-4b4b-9fed-a66edcc6c70e admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:42.955703 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a76078-a8f9-4b4b-9fed-a66edcc6c70e admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:42.961691 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914f3da-c281-4097-8a36-77ca1a3a593a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:42.962800 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914f3da-c281-4097-8a36-77ca1a3a593a admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:42.963908 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914f3da-c281-4097-8a36-77ca1a3a593a admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:42.964499 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914f3da-c281-4097-8a36-77ca1a3a593a admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:42.965884 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914f3da-c281-4097-8a36-77ca1a3a593a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:42.968351 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914f3da-c281-4097-8a36-77ca1a3a593a admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:42.984275 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1371/2763] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:15:42.988630 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a90c3d-697f-4c61-9a04-c0c4fba99713 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:43.003113 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a90c3d-697f-4c61-9a04-c0c4fba99713 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:43.010085 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcd651-1616-4a67-9bd9-c24741d28560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:43.011026 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcd651-1616-4a67-9bd9-c24741d28560 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:43.011316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcd651-1616-4a67-9bd9-c24741d28560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:43.011595 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcd651-1616-4a67-9bd9-c24741d28560 admin admin] PATH_INFO: `/v3/projects/a44968d7c3564d76b788d342eb7ff502/users/51e4d45647a14b78b0f134e4598a2a90/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:43.044834 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcd651-1616-4a67-9bd9-c24741d28560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a44968d7c3564d76b788d342eb7ff502, user_id=51e4d45647a14b78b0f134e4598a2a90, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:43.047705 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcd651-1616-4a67-9bd9-c24741d28560 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:43.065659 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1393/2764] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:15:42 2022] PUT /identity/v3/projects/a44968d7c3564d76b788d342eb7ff502/users/51e4d45647a14b78b0f134e4598a2a90/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:15:43.075724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2855887-3ebc-40ed-afd1-9c70fe81e805 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:43.076290 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2855887-3ebc-40ed-afd1-9c70fe81e805 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:43.076969 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2855887-3ebc-40ed-afd1-9c70fe81e805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:43.118005 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-f2855887-3ebc-40ed-afd1-9c70fe81e805 None None] MFA Rules not processed for user `51e4d45647a14b78b0f134e4598a2a90`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:15:43.127802 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f2855887-3ebc-40ed-afd1-9c70fe81e805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:43.257498 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1372/2765] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:15:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:15:43.280549 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcd651-1616-4a67-9bd9-c24741d28560 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:43.282720 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d9bcd651-1616-4a67-9bd9-c24741d28560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:43.447169 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcd651-1616-4a67-9bd9-c24741d28560 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:43.463514 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291cde0-53e0-47c5-8f6f-49272634254f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:43.464747 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291cde0-53e0-47c5-8f6f-49272634254f service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:43.465547 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291cde0-53e0-47c5-8f6f-49272634254f service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:43.466176 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291cde0-53e0-47c5-8f6f-49272634254f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:43.494744 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1291cde0-53e0-47c5-8f6f-49272634254f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:43.498153 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1291cde0-53e0-47c5-8f6f-49272634254f service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:43.520222 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1394/2766] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:15:43 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:15:48.139476 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2855887-3ebc-40ed-afd1-9c70fe81e805 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:48.142888 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f2855887-3ebc-40ed-afd1-9c70fe81e805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:48.200170 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:15:48.202396 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291cde0-53e0-47c5-8f6f-49272634254f service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:48.204457 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1291cde0-53e0-47c5-8f6f-49272634254f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:48.221047 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2855887-3ebc-40ed-afd1-9c70fe81e805 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:48.226154 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230c236-ef66-4bbc-8943-fbc05b6bd967 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '596f126382e3440fb3cf7c357eef889f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:48.226849 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230c236-ef66-4bbc-8943-fbc05b6bd967 service placement] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:48.227322 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230c236-ef66-4bbc-8943-fbc05b6bd967 service placement] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:48.227606 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230c236-ef66-4bbc-8943-fbc05b6bd967 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:48.229397 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f230c236-ef66-4bbc-8943-fbc05b6bd967 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:48.243762 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f230c236-ef66-4bbc-8943-fbc05b6bd967 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:48.245387 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f230c236-ef66-4bbc-8943-fbc05b6bd967 service placement] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:48.257685 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1373/2767] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:15:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:15:48.280371 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291cde0-53e0-47c5-8f6f-49272634254f service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:48.285885 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379aa423-992c-40e3-a057-0d2e184383ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:48.287072 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379aa423-992c-40e3-a057-0d2e184383ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:48.287369 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379aa423-992c-40e3-a057-0d2e184383ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:48.287708 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379aa423-992c-40e3-a057-0d2e184383ba admin admin] PATH_INFO: `/v3/users/42c96a85400941818b658cca8cb56fec` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:48.289601 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379aa423-992c-40e3-a057-0d2e184383ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=42c96a85400941818b658cca8cb56fec)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:48.291489 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379aa423-992c-40e3-a057-0d2e184383ba admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:48.306501 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:15:48.345130 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-379aa423-992c-40e3-a057-0d2e184383ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42c96a85400941818b658cca8cb56fec'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:48.352995 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-379aa423-992c-40e3-a057-0d2e184383ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42c96a85400941818b658cca8cb56fec'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:48.372974 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-379aa423-992c-40e3-a057-0d2e184383ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42c96a85400941818b658cca8cb56fec'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:48.372974 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-379aa423-992c-40e3-a057-0d2e184383ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42c96a85400941818b658cca8cb56fec'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:48.372974 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-379aa423-992c-40e3-a057-0d2e184383ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42c96a85400941818b658cca8cb56fec'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:48.386948 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1395/2768] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:15:48 2022] DELETE /identity/v3/users/42c96a85400941818b658cca8cb56fec => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:15:48.577184 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230c236-ef66-4bbc-8943-fbc05b6bd967 service placement] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:48.588345 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230c236-ef66-4bbc-8943-fbc05b6bd967 service placement] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:48.661116 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d478d0-7780-4596-b854-f834136de652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:48.662048 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d478d0-7780-4596-b854-f834136de652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:48.662374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d478d0-7780-4596-b854-f834136de652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:48.662628 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d478d0-7780-4596-b854-f834136de652 admin admin] PATH_INFO: `/v3/projects/45d94db7075c40cbae3f106a9661d969` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:48.663820 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d478d0-7780-4596-b854-f834136de652 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45d94db7075c40cbae3f106a9661d969)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:48.665726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d478d0-7780-4596-b854-f834136de652 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:48.715840 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-45d478d0-7780-4596-b854-f834136de652 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45d94db7075c40cbae3f106a9661d969', 'request_id': 'req-45d478d0-7780-4596-b854-f834136de652'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:48.724811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-45d478d0-7780-4596-b854-f834136de652 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45d94db7075c40cbae3f106a9661d969', 'request_id': 'req-45d478d0-7780-4596-b854-f834136de652'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:48.731328 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-45d478d0-7780-4596-b854-f834136de652 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45d94db7075c40cbae3f106a9661d969', 'request_id': 'req-45d478d0-7780-4596-b854-f834136de652'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:48.746250 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-45d478d0-7780-4596-b854-f834136de652 admin admin] The token cache is being invalidate because project 45d94db7075c40cbae3f106a9661d969 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:15:48.746541 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-45d478d0-7780-4596-b854-f834136de652 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:48.747193 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-45d478d0-7780-4596-b854-f834136de652 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:48.748861 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1374/2769] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:15:48 2022] DELETE /identity/v3/projects/45d94db7075c40cbae3f106a9661d969 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:15:48.753466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379aa423-992c-40e3-a057-0d2e184383ba admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:48.754947 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-379aa423-992c-40e3-a057-0d2e184383ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:48.769429 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379aa423-992c-40e3-a057-0d2e184383ba admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:48.828423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34687e-256d-4d1b-b389-6c038d6c4194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:48.829795 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34687e-256d-4d1b-b389-6c038d6c4194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:48.830079 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34687e-256d-4d1b-b389-6c038d6c4194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:48.830388 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34687e-256d-4d1b-b389-6c038d6c4194 admin admin] PATH_INFO: `/v3/users/9b89b641099e4132817893abbab6bf99` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:48.831759 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34687e-256d-4d1b-b389-6c038d6c4194 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b89b641099e4132817893abbab6bf99)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:48.833707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34687e-256d-4d1b-b389-6c038d6c4194 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:48.882034 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3c34687e-256d-4d1b-b389-6c038d6c4194 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b89b641099e4132817893abbab6bf99'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:48.889869 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3c34687e-256d-4d1b-b389-6c038d6c4194 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b89b641099e4132817893abbab6bf99'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:48.904331 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3c34687e-256d-4d1b-b389-6c038d6c4194 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b89b641099e4132817893abbab6bf99'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:48.905012 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3c34687e-256d-4d1b-b389-6c038d6c4194 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b89b641099e4132817893abbab6bf99'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:48.905851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3c34687e-256d-4d1b-b389-6c038d6c4194 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b89b641099e4132817893abbab6bf99'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:48.920318 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1396/2770] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:15:48 2022] DELETE /identity/v3/users/9b89b641099e4132817893abbab6bf99 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:15:49.101375 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d478d0-7780-4596-b854-f834136de652 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:49.103062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-45d478d0-7780-4596-b854-f834136de652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:49.117646 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d478d0-7780-4596-b854-f834136de652 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:49.176668 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d7e7c-3ad8-4472-9544-94ba9c744472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:49.177535 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d7e7c-3ad8-4472-9544-94ba9c744472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:49.177818 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d7e7c-3ad8-4472-9544-94ba9c744472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:49.178099 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d7e7c-3ad8-4472-9544-94ba9c744472 admin admin] PATH_INFO: `/v3/projects/d380d672826643ba9d6f491c3a981dff` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:49.179306 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d7e7c-3ad8-4472-9544-94ba9c744472 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d380d672826643ba9d6f491c3a981dff)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:49.181202 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d7e7c-3ad8-4472-9544-94ba9c744472 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:49.192964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:15:49.229551 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-8e4d7e7c-3ad8-4472-9544-94ba9c744472 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd380d672826643ba9d6f491c3a981dff', 'request_id': 'req-8e4d7e7c-3ad8-4472-9544-94ba9c744472'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:49.237146 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-8e4d7e7c-3ad8-4472-9544-94ba9c744472 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd380d672826643ba9d6f491c3a981dff', 'request_id': 'req-8e4d7e7c-3ad8-4472-9544-94ba9c744472'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:49.243229 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-8e4d7e7c-3ad8-4472-9544-94ba9c744472 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd380d672826643ba9d6f491c3a981dff', 'request_id': 'req-8e4d7e7c-3ad8-4472-9544-94ba9c744472'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:49.254658 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-8e4d7e7c-3ad8-4472-9544-94ba9c744472 admin admin] The token cache is being invalidate because project d380d672826643ba9d6f491c3a981dff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:15:49.254942 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-8e4d7e7c-3ad8-4472-9544-94ba9c744472 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:49.255598 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-8e4d7e7c-3ad8-4472-9544-94ba9c744472 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:49.257144 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1375/2771] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:15:49 2022] DELETE /identity/v3/projects/d380d672826643ba9d6f491c3a981dff => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:15:50.916635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34687e-256d-4d1b-b389-6c038d6c4194 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:50.918077 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3c34687e-256d-4d1b-b389-6c038d6c4194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:50.931717 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34687e-256d-4d1b-b389-6c038d6c4194 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:50.939679 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:15:50.993657 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c3c1a-63e4-4440-be11-a3ec3fe6f669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:50.994574 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c3c1a-63e4-4440-be11-a3ec3fe6f669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:50.994862 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c3c1a-63e4-4440-be11-a3ec3fe6f669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:50.995168 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c3c1a-63e4-4440-be11-a3ec3fe6f669 admin admin] PATH_INFO: `/v3/users/51e4d45647a14b78b0f134e4598a2a90` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:50.996342 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c3c1a-63e4-4440-be11-a3ec3fe6f669 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51e4d45647a14b78b0f134e4598a2a90)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:50.998324 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c3c1a-63e4-4440-be11-a3ec3fe6f669 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:51.049461 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-012c3c1a-63e4-4440-be11-a3ec3fe6f669 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51e4d45647a14b78b0f134e4598a2a90'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:51.057127 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-012c3c1a-63e4-4440-be11-a3ec3fe6f669 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51e4d45647a14b78b0f134e4598a2a90'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:51.069408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-012c3c1a-63e4-4440-be11-a3ec3fe6f669 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51e4d45647a14b78b0f134e4598a2a90'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:51.070073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-012c3c1a-63e4-4440-be11-a3ec3fe6f669 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51e4d45647a14b78b0f134e4598a2a90'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:51.070719 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-012c3c1a-63e4-4440-be11-a3ec3fe6f669 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51e4d45647a14b78b0f134e4598a2a90'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:51.083553 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1397/2772] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:15:50 2022] DELETE /identity/v3/users/51e4d45647a14b78b0f134e4598a2a90 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:15:51.087970 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0f058-d030-4d04-84da-0b2a7f00e924 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:51.088429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0f058-d030-4d04-84da-0b2a7f00e924 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:51.088547 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0f058-d030-4d04-84da-0b2a7f00e924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:51.106707 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-61c0f058-d030-4d04-84da-0b2a7f00e924 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:15:51.107901 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-61c0f058-d030-4d04-84da-0b2a7f00e924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:51.167274 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1376/2773] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:15:51.176947 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c3c1a-63e4-4440-be11-a3ec3fe6f669 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:51.178452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-012c3c1a-63e4-4440-be11-a3ec3fe6f669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:51.236329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c3c1a-63e4-4440-be11-a3ec3fe6f669 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:51.240026 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c6542-8eb0-44f0-8f3e-4ccb1b55be73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:51.240678 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c6542-8eb0-44f0-8f3e-4ccb1b55be73 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:51.240969 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c6542-8eb0-44f0-8f3e-4ccb1b55be73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:51.241249 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c6542-8eb0-44f0-8f3e-4ccb1b55be73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:51.250502 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c6542-8eb0-44f0-8f3e-4ccb1b55be73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:51.252055 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c6542-8eb0-44f0-8f3e-4ccb1b55be73 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:51.261834 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1398/2774] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:15:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:15:51.547678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0f058-d030-4d04-84da-0b2a7f00e924 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:51.549479 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-61c0f058-d030-4d04-84da-0b2a7f00e924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:51.562588 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0f058-d030-4d04-84da-0b2a7f00e924 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:51.568292 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9da34d-1332-48ad-b421-638134f8bb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:51.569307 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9da34d-1332-48ad-b421-638134f8bb63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:51.569680 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9da34d-1332-48ad-b421-638134f8bb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:51.570031 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9da34d-1332-48ad-b421-638134f8bb63 admin admin] PATH_INFO: `/v3/projects/a44968d7c3564d76b788d342eb7ff502` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:51.571382 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9da34d-1332-48ad-b421-638134f8bb63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a44968d7c3564d76b788d342eb7ff502)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:51.573388 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9da34d-1332-48ad-b421-638134f8bb63 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:51.624651 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6b9da34d-1332-48ad-b421-638134f8bb63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a44968d7c3564d76b788d342eb7ff502', 'request_id': 'req-6b9da34d-1332-48ad-b421-638134f8bb63'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:51.634657 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6b9da34d-1332-48ad-b421-638134f8bb63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a44968d7c3564d76b788d342eb7ff502', 'request_id': 'req-6b9da34d-1332-48ad-b421-638134f8bb63'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:51.642545 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6b9da34d-1332-48ad-b421-638134f8bb63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a44968d7c3564d76b788d342eb7ff502', 'request_id': 'req-6b9da34d-1332-48ad-b421-638134f8bb63'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:51.655535 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6b9da34d-1332-48ad-b421-638134f8bb63 admin admin] The token cache is being invalidate because project a44968d7c3564d76b788d342eb7ff502 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:15:51.655810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6b9da34d-1332-48ad-b421-638134f8bb63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:51.656533 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6b9da34d-1332-48ad-b421-638134f8bb63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:15:51.658639 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1377/2775] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:15:51 2022] DELETE /identity/v3/projects/a44968d7c3564d76b788d342eb7ff502 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:15:51.804431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c6542-8eb0-44f0-8f3e-4ccb1b55be73 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:51.805977 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f46c6542-8eb0-44f0-8f3e-4ccb1b55be73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:51.852713 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:15:51.899910 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c6542-8eb0-44f0-8f3e-4ccb1b55be73 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:51.974098 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff1fb4-b482-4d17-b069-4095089c51a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1899c317083d40f5b490cd9eb63fe521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:51.974812 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff1fb4-b482-4d17-b069-4095089c51a0 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:51.974971 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff1fb4-b482-4d17-b069-4095089c51a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:51.975246 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff1fb4-b482-4d17-b069-4095089c51a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:51.977030 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-adff1fb4-b482-4d17-b069-4095089c51a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:52.039632 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff1fb4-b482-4d17-b069-4095089c51a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:52.041357 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff1fb4-b482-4d17-b069-4095089c51a0 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:52.053233 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1399/2776] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:15:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:15:54.909084 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8896a0a-6ca6-4ef8-ba10-f67bc44c9e61 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:54.909482 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8896a0a-6ca6-4ef8-ba10-f67bc44c9e61 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:54.909482 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8896a0a-6ca6-4ef8-ba10-f67bc44c9e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:54.927887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-e8896a0a-6ca6-4ef8-ba10-f67bc44c9e61 None None] MFA Rules not processed for user `3e06d1b0e1a9472db764f4be7e42fdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:15:54.929079 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-e8896a0a-6ca6-4ef8-ba10-f67bc44c9e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:54.948622 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:15:54.988287 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1378/2777] 10.222.0.76 () {58 vars in 967 bytes} [Thu May 26 18:15:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:15:55.000916 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff1fb4-b482-4d17-b069-4095089c51a0 service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:15:55.003008 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-adff1fb4-b482-4d17-b069-4095089c51a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:15:55.073022 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff1fb4-b482-4d17-b069-4095089c51a0 service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:15:55.078316 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe79f8d-3dfc-4f56-b47a-49c1e02d593d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ca774e864ca44cfb187e38bc8a8f6ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:15:55.078965 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe79f8d-3dfc-4f56-b47a-49c1e02d593d service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:15:55.079250 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe79f8d-3dfc-4f56-b47a-49c1e02d593d service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:15:55.079555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe79f8d-3dfc-4f56-b47a-49c1e02d593d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:15:55.087949 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe79f8d-3dfc-4f56-b47a-49c1e02d593d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:15:55.089535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe79f8d-3dfc-4f56-b47a-49c1e02d593d service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:15:55.092998 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:15:55.099096 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1400/2778] 10.222.0.76 () {60 vars in 1384 bytes} [Thu May 26 18:15:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 18:16:04.980809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=14 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:05.100291 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:35.351375 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8896a0a-6ca6-4ef8-ba10-f67bc44c9e61 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:35.352822 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-e8896a0a-6ca6-4ef8-ba10-f67bc44c9e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:35.458681 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8896a0a-6ca6-4ef8-ba10-f67bc44c9e61 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:35.462436 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3278d8f-9437-4ce6-ab67-5bb7af707fba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:35.463089 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3278d8f-9437-4ce6-ab67-5bb7af707fba service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:35.463374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3278d8f-9437-4ce6-ab67-5bb7af707fba service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:35.463657 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3278d8f-9437-4ce6-ab67-5bb7af707fba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:35.465427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-b3278d8f-9437-4ce6-ab67-5bb7af707fba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:35.571037 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3278d8f-9437-4ce6-ab67-5bb7af707fba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:35.572428 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3278d8f-9437-4ce6-ab67-5bb7af707fba service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:35.592460 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1379/2779] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:16:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:16:38.766456 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe79f8d-3dfc-4f56-b47a-49c1e02d593d service swift] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:38.768524 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cbe79f8d-3dfc-4f56-b47a-49c1e02d593d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:38.808588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe79f8d-3dfc-4f56-b47a-49c1e02d593d service swift] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:38.813121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72097bfe-ba13-4e43-911e-18e61aedc00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:38.813996 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72097bfe-ba13-4e43-911e-18e61aedc00d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:38.814331 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72097bfe-ba13-4e43-911e-18e61aedc00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:38.814608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72097bfe-ba13-4e43-911e-18e61aedc00d admin admin] PATH_INFO: `/v3/users/1b262e67242c4384ba5bf0356a08679e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:38.815794 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72097bfe-ba13-4e43-911e-18e61aedc00d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b262e67242c4384ba5bf0356a08679e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:38.817789 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72097bfe-ba13-4e43-911e-18e61aedc00d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:38.866048 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-72097bfe-ba13-4e43-911e-18e61aedc00d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b262e67242c4384ba5bf0356a08679e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:38.873833 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-72097bfe-ba13-4e43-911e-18e61aedc00d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b262e67242c4384ba5bf0356a08679e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:38.886083 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-72097bfe-ba13-4e43-911e-18e61aedc00d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b262e67242c4384ba5bf0356a08679e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:38.886741 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-72097bfe-ba13-4e43-911e-18e61aedc00d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b262e67242c4384ba5bf0356a08679e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:38.887346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-72097bfe-ba13-4e43-911e-18e61aedc00d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b262e67242c4384ba5bf0356a08679e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:38.900635 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1401/2780] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:16:38 2022] DELETE /identity/v3/users/1b262e67242c4384ba5bf0356a08679e => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:39.090947 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3278d8f-9437-4ce6-ab67-5bb7af707fba service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:39.102383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3278d8f-9437-4ce6-ab67-5bb7af707fba service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:39.112848 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=100 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:39.164942 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16feb79a-9e61-4947-a7e6-7c9b46c717cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:39.165809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16feb79a-9e61-4947-a7e6-7c9b46c717cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:39.166128 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16feb79a-9e61-4947-a7e6-7c9b46c717cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:39.166440 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16feb79a-9e61-4947-a7e6-7c9b46c717cb admin admin] PATH_INFO: `/v3/projects/1b13e82b890d4bedbd8ecb6bcbd603c6` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:39.167639 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16feb79a-9e61-4947-a7e6-7c9b46c717cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b13e82b890d4bedbd8ecb6bcbd603c6)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:39.169576 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16feb79a-9e61-4947-a7e6-7c9b46c717cb admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:39.216260 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-16feb79a-9e61-4947-a7e6-7c9b46c717cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b13e82b890d4bedbd8ecb6bcbd603c6', 'request_id': 'req-16feb79a-9e61-4947-a7e6-7c9b46c717cb'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:39.223586 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-16feb79a-9e61-4947-a7e6-7c9b46c717cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b13e82b890d4bedbd8ecb6bcbd603c6', 'request_id': 'req-16feb79a-9e61-4947-a7e6-7c9b46c717cb'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:39.229521 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-16feb79a-9e61-4947-a7e6-7c9b46c717cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b13e82b890d4bedbd8ecb6bcbd603c6', 'request_id': 'req-16feb79a-9e61-4947-a7e6-7c9b46c717cb'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:39.240472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-16feb79a-9e61-4947-a7e6-7c9b46c717cb admin admin] The token cache is being invalidate because project 1b13e82b890d4bedbd8ecb6bcbd603c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:16:39.240758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-16feb79a-9e61-4947-a7e6-7c9b46c717cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:39.241396 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-16feb79a-9e61-4947-a7e6-7c9b46c717cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:39.243104 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1380/2781] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:16:39 2022] DELETE /identity/v3/projects/1b13e82b890d4bedbd8ecb6bcbd603c6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:39.248067 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246104ef-e6cb-4d1d-97e1-94fe1e5633bb None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:39.248425 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246104ef-e6cb-4d1d-97e1-94fe1e5633bb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:39.248711 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246104ef-e6cb-4d1d-97e1-94fe1e5633bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:39.296282 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-246104ef-e6cb-4d1d-97e1-94fe1e5633bb None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:39.297494 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-246104ef-e6cb-4d1d-97e1-94fe1e5633bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:39.357263 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1402/2782] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:16:39.365456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bdb9dc-3219-4088-aae8-606cb7ad1227 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:39.365811 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bdb9dc-3219-4088-aae8-606cb7ad1227 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:39.366140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bdb9dc-3219-4088-aae8-606cb7ad1227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:39.385013 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-93bdb9dc-3219-4088-aae8-606cb7ad1227 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:39.386350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-93bdb9dc-3219-4088-aae8-606cb7ad1227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:39.394256 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1381/2783] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:39.399812 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246104ef-e6cb-4d1d-97e1-94fe1e5633bb None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:39.409540 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246104ef-e6cb-4d1d-97e1-94fe1e5633bb None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:39.414755 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87db01e-3ec8-4884-af61-65d8286e60c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:39.415412 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87db01e-3ec8-4884-af61-65d8286e60c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:39.415701 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87db01e-3ec8-4884-af61-65d8286e60c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:39.415985 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87db01e-3ec8-4884-af61-65d8286e60c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:39.416714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87db01e-3ec8-4884-af61-65d8286e60c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:16:39.417031 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87db01e-3ec8-4884-af61-65d8286e60c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:39.418670 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87db01e-3ec8-4884-af61-65d8286e60c6 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:39.430516 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1403/2784] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:16:39.434849 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bdb9dc-3219-4088-aae8-606cb7ad1227 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:39.444311 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bdb9dc-3219-4088-aae8-606cb7ad1227 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:39.450797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93d7a1-57bb-45aa-a4aa-afaaddef39b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:39.451453 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93d7a1-57bb-45aa-a4aa-afaaddef39b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:39.451739 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93d7a1-57bb-45aa-a4aa-afaaddef39b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:39.452020 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93d7a1-57bb-45aa-a4aa-afaaddef39b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:39.452727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93d7a1-57bb-45aa-a4aa-afaaddef39b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:39.454674 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93d7a1-57bb-45aa-a4aa-afaaddef39b8 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:39.480748 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1382/2785] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:16:39 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:16:39.485011 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87db01e-3ec8-4884-af61-65d8286e60c6 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:39.495003 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87db01e-3ec8-4884-af61-65d8286e60c6 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:39.503838 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:39.555096 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaec145-4a23-45bc-b6cf-809ada1cd58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:39.555763 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaec145-4a23-45bc-b6cf-809ada1cd58c admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:39.556053 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaec145-4a23-45bc-b6cf-809ada1cd58c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:39.556340 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaec145-4a23-45bc-b6cf-809ada1cd58c admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:39.557040 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaec145-4a23-45bc-b6cf-809ada1cd58c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:39.559010 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaec145-4a23-45bc-b6cf-809ada1cd58c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:39.579020 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1404/2786] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:16:39 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:16:39.583500 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93d7a1-57bb-45aa-a4aa-afaaddef39b8 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:39.594270 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93d7a1-57bb-45aa-a4aa-afaaddef39b8 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:39.598880 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b7755-9027-4374-91c7-f768200a3d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:39.600140 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b7755-9027-4374-91c7-f768200a3d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:39.600428 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b7755-9027-4374-91c7-f768200a3d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:39.600827 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b7755-9027-4374-91c7-f768200a3d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:39.601560 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b7755-9027-4374-91c7-f768200a3d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:39.603193 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b7755-9027-4374-91c7-f768200a3d4d admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:39.614061 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1383/2787] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:16:39.618001 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaec145-4a23-45bc-b6cf-809ada1cd58c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:39.627871 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaec145-4a23-45bc-b6cf-809ada1cd58c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:39.632739 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46a9f1-0741-437d-b613-6b11d47dbeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:39.633390 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46a9f1-0741-437d-b613-6b11d47dbeee admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:39.633675 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46a9f1-0741-437d-b613-6b11d47dbeee admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:39.633946 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46a9f1-0741-437d-b613-6b11d47dbeee admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:39.634704 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46a9f1-0741-437d-b613-6b11d47dbeee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:39.636303 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46a9f1-0741-437d-b613-6b11d47dbeee admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:39.647172 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1405/2788] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:16:39.651715 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b7755-9027-4374-91c7-f768200a3d4d admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:39.661544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b7755-9027-4374-91c7-f768200a3d4d admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:39.666761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abcd04-30ae-4da5-af82-9e4628401ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:39.667817 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abcd04-30ae-4da5-af82-9e4628401ca5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:39.668215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abcd04-30ae-4da5-af82-9e4628401ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:39.668497 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abcd04-30ae-4da5-af82-9e4628401ca5 admin admin] PATH_INFO: `/v3/projects/7b19b5ca8bad4813a6cf82f720ef18e5/users/82459a63bc7d41a88e5b3ab8824a0e55/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:39.683532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2abcd04-30ae-4da5-af82-9e4628401ca5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b19b5ca8bad4813a6cf82f720ef18e5, user_id=82459a63bc7d41a88e5b3ab8824a0e55, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:39.686165 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2abcd04-30ae-4da5-af82-9e4628401ca5 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:39.699920 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1384/2789] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:16:39 2022] PUT /identity/v3/projects/7b19b5ca8bad4813a6cf82f720ef18e5/users/82459a63bc7d41a88e5b3ab8824a0e55/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:39.712082 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bb0e2-b8db-4890-84e9-27002159d3a1 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:39.712371 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bb0e2-b8db-4890-84e9-27002159d3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:39.712650 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bb0e2-b8db-4890-84e9-27002159d3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:39.739001 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2d5bb0e2-b8db-4890-84e9-27002159d3a1 None None] MFA Rules not processed for user `82459a63bc7d41a88e5b3ab8824a0e55`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:39.740803 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2d5bb0e2-b8db-4890-84e9-27002159d3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:39.810580 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1406/2790] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:39.815171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abcd04-30ae-4da5-af82-9e4628401ca5 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:39.859453 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:39.879907 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abcd04-30ae-4da5-af82-9e4628401ca5 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:39.885524 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1337c-aa96-4a05-af5b-00b2d5b2c921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:39.886179 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1337c-aa96-4a05-af5b-00b2d5b2c921 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:39.886490 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1337c-aa96-4a05-af5b-00b2d5b2c921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:39.886769 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1337c-aa96-4a05-af5b-00b2d5b2c921 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:39.887472 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1337c-aa96-4a05-af5b-00b2d5b2c921 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:39.889389 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1337c-aa96-4a05-af5b-00b2d5b2c921 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:39.915781 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1385/2791] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:16:39 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:16:39.919887 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bb0e2-b8db-4890-84e9-27002159d3a1 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:39.931344 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bb0e2-b8db-4890-84e9-27002159d3a1 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:39.988792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5c0de-85f1-4efa-91ea-28faec079c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:39.989455 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5c0de-85f1-4efa-91ea-28faec079c1b admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:39.989743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5c0de-85f1-4efa-91ea-28faec079c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:39.990055 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5c0de-85f1-4efa-91ea-28faec079c1b admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:39.991184 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5c0de-85f1-4efa-91ea-28faec079c1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:39.993090 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5c0de-85f1-4efa-91ea-28faec079c1b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:40.013454 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1407/2792] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:16:39 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:16:40.017516 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1337c-aa96-4a05-af5b-00b2d5b2c921 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:40.028774 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1337c-aa96-4a05-af5b-00b2d5b2c921 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:40.033438 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41ae37-4578-44b9-b141-0c5f764d158f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:40.034102 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41ae37-4578-44b9-b141-0c5f764d158f admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:40.034428 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41ae37-4578-44b9-b141-0c5f764d158f admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:40.034711 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41ae37-4578-44b9-b141-0c5f764d158f admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:40.035434 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41ae37-4578-44b9-b141-0c5f764d158f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:40.037241 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41ae37-4578-44b9-b141-0c5f764d158f admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:40.048064 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1386/2793] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:16:40.052210 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5c0de-85f1-4efa-91ea-28faec079c1b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:40.062276 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5c0de-85f1-4efa-91ea-28faec079c1b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:40.066851 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e771114-dfad-4f91-96c6-7de0d3a55c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:40.067709 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e771114-dfad-4f91-96c6-7de0d3a55c5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:40.068489 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e771114-dfad-4f91-96c6-7de0d3a55c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:40.068796 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e771114-dfad-4f91-96c6-7de0d3a55c5e admin admin] PATH_INFO: `/v3/projects/bee4302010774fd682f6488542c50530/users/e3b9f0f3d0214704a573f09a740f850e/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:40.077473 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:40.086038 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e771114-dfad-4f91-96c6-7de0d3a55c5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bee4302010774fd682f6488542c50530, user_id=e3b9f0f3d0214704a573f09a740f850e, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:40.088776 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e771114-dfad-4f91-96c6-7de0d3a55c5e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:40.101670 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1408/2794] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:16:40 2022] PUT /identity/v3/projects/bee4302010774fd682f6488542c50530/users/e3b9f0f3d0214704a573f09a740f850e/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:40.105828 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41ae37-4578-44b9-b141-0c5f764d158f admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:40.116379 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41ae37-4578-44b9-b141-0c5f764d158f admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:40.174756 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c222a7a7-1017-4b32-bdb1-6f3e19a6215c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:40.175429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222a7a7-1017-4b32-bdb1-6f3e19a6215c admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:40.175710 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222a7a7-1017-4b32-bdb1-6f3e19a6215c admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:40.175984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222a7a7-1017-4b32-bdb1-6f3e19a6215c admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:40.176693 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c222a7a7-1017-4b32-bdb1-6f3e19a6215c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:40.179362 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c222a7a7-1017-4b32-bdb1-6f3e19a6215c admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:40.189592 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1387/2795] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:16:40.194813 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e771114-dfad-4f91-96c6-7de0d3a55c5e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:40.205275 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e771114-dfad-4f91-96c6-7de0d3a55c5e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:40.210637 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b29142-8c05-4a16-8b31-6c0dd458f79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:40.211531 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b29142-8c05-4a16-8b31-6c0dd458f79f admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:40.211820 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b29142-8c05-4a16-8b31-6c0dd458f79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:40.213441 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b29142-8c05-4a16-8b31-6c0dd458f79f admin admin] PATH_INFO: `/v3/projects/bee4302010774fd682f6488542c50530/users/e3b9f0f3d0214704a573f09a740f850e/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:40.214360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b29142-8c05-4a16-8b31-6c0dd458f79f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bee4302010774fd682f6488542c50530, user_id=e3b9f0f3d0214704a573f09a740f850e, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:40.217065 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b29142-8c05-4a16-8b31-6c0dd458f79f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:40.229545 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1409/2796] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:16:40 2022] PUT /identity/v3/projects/bee4302010774fd682f6488542c50530/users/e3b9f0f3d0214704a573f09a740f850e/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:40.239814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a1b40-478c-4870-8d65-07be163fac22 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:40.240794 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a1b40-478c-4870-8d65-07be163fac22 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:40.241259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a1b40-478c-4870-8d65-07be163fac22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:40.268352 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-540a1b40-478c-4870-8d65-07be163fac22 None None] MFA Rules not processed for user `e3b9f0f3d0214704a573f09a740f850e`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:40.269554 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-540a1b40-478c-4870-8d65-07be163fac22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:40.341383 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1388/2797] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:40.352637 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b29142-8c05-4a16-8b31-6c0dd458f79f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:40.354363 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a4b29142-8c05-4a16-8b31-6c0dd458f79f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:40.413534 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b29142-8c05-4a16-8b31-6c0dd458f79f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:40.417067 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633801a-0c34-4738-bd99-ae80e5386103 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:40.417705 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633801a-0c34-4738-bd99-ae80e5386103 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:40.417985 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633801a-0c34-4738-bd99-ae80e5386103 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:40.418285 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633801a-0c34-4738-bd99-ae80e5386103 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:40.427271 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633801a-0c34-4738-bd99-ae80e5386103 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:40.429518 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633801a-0c34-4738-bd99-ae80e5386103 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:40.439214 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1410/2798] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:16:40 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:16:40.815687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a1b40-478c-4870-8d65-07be163fac22 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:40.856571 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:40.881282 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a1b40-478c-4870-8d65-07be163fac22 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:40.885963 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563cf63-8960-487e-b1bc-c979a0dd78d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:40.886878 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563cf63-8960-487e-b1bc-c979a0dd78d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:40.887163 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563cf63-8960-487e-b1bc-c979a0dd78d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:40.887435 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563cf63-8960-487e-b1bc-c979a0dd78d9 admin admin] PATH_INFO: `/v3/users/82459a63bc7d41a88e5b3ab8824a0e55` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:40.888621 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563cf63-8960-487e-b1bc-c979a0dd78d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82459a63bc7d41a88e5b3ab8824a0e55)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:40.890569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563cf63-8960-487e-b1bc-c979a0dd78d9 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:40.936235 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6563cf63-8960-487e-b1bc-c979a0dd78d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82459a63bc7d41a88e5b3ab8824a0e55'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:40.944608 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6563cf63-8960-487e-b1bc-c979a0dd78d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82459a63bc7d41a88e5b3ab8824a0e55'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:40.945227 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6563cf63-8960-487e-b1bc-c979a0dd78d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82459a63bc7d41a88e5b3ab8824a0e55'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:40.945833 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6563cf63-8960-487e-b1bc-c979a0dd78d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82459a63bc7d41a88e5b3ab8824a0e55'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:40.953160 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6563cf63-8960-487e-b1bc-c979a0dd78d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82459a63bc7d41a88e5b3ab8824a0e55'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:40.970856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1389/2799] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:16:40 2022] DELETE /identity/v3/users/82459a63bc7d41a88e5b3ab8824a0e55 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:40.975495 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49862f9b-253e-489d-bb4c-41abcc060653 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:40.975994 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49862f9b-253e-489d-bb4c-41abcc060653 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:40.976082 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49862f9b-253e-489d-bb4c-41abcc060653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:40.993964 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-49862f9b-253e-489d-bb4c-41abcc060653 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:40.995249 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-49862f9b-253e-489d-bb4c-41abcc060653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:41.055095 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1411/2800] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:41.064888 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563cf63-8960-487e-b1bc-c979a0dd78d9 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:41.066338 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-6563cf63-8960-487e-b1bc-c979a0dd78d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:41.124666 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563cf63-8960-487e-b1bc-c979a0dd78d9 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:41.128462 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd8a69-27f9-4cab-b1ec-6bdaff901061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:41.129123 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd8a69-27f9-4cab-b1ec-6bdaff901061 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:41.129411 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd8a69-27f9-4cab-b1ec-6bdaff901061 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:41.129695 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd8a69-27f9-4cab-b1ec-6bdaff901061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:41.140631 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd8a69-27f9-4cab-b1ec-6bdaff901061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:41.142121 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd8a69-27f9-4cab-b1ec-6bdaff901061 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:41.152642 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1390/2801] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:16:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:16:41.428940 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49862f9b-253e-489d-bb4c-41abcc060653 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:41.430591 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-49862f9b-253e-489d-bb4c-41abcc060653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:41.444638 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49862f9b-253e-489d-bb4c-41abcc060653 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:41.449720 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a802ab-6433-428b-bf25-c40eaa4776cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:41.450626 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a802ab-6433-428b-bf25-c40eaa4776cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:41.450916 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a802ab-6433-428b-bf25-c40eaa4776cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:41.451195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a802ab-6433-428b-bf25-c40eaa4776cc admin admin] PATH_INFO: `/v3/projects/7b19b5ca8bad4813a6cf82f720ef18e5` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:41.452390 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a802ab-6433-428b-bf25-c40eaa4776cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b19b5ca8bad4813a6cf82f720ef18e5)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:41.454346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a802ab-6433-428b-bf25-c40eaa4776cc admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:41.502116 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b9a802ab-6433-428b-bf25-c40eaa4776cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b19b5ca8bad4813a6cf82f720ef18e5', 'request_id': 'req-b9a802ab-6433-428b-bf25-c40eaa4776cc'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:41.507532 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b9a802ab-6433-428b-bf25-c40eaa4776cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b19b5ca8bad4813a6cf82f720ef18e5', 'request_id': 'req-b9a802ab-6433-428b-bf25-c40eaa4776cc'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:41.514042 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b9a802ab-6433-428b-bf25-c40eaa4776cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b19b5ca8bad4813a6cf82f720ef18e5', 'request_id': 'req-b9a802ab-6433-428b-bf25-c40eaa4776cc'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:41.524192 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:41.529930 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b9a802ab-6433-428b-bf25-c40eaa4776cc admin admin] The token cache is being invalidate because project 7b19b5ca8bad4813a6cf82f720ef18e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:16:41.530519 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b9a802ab-6433-428b-bf25-c40eaa4776cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:41.531346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b9a802ab-6433-428b-bf25-c40eaa4776cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:41.533768 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1412/2802] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:16:41 2022] DELETE /identity/v3/projects/7b19b5ca8bad4813a6cf82f720ef18e5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:41.538470 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd8a69-27f9-4cab-b1ec-6bdaff901061 service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:41.548630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd8a69-27f9-4cab-b1ec-6bdaff901061 service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:41.601698 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:41.608984 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64a061-6aef-4548-a0c4-2b1f834d222a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:41.609954 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64a061-6aef-4548-a0c4-2b1f834d222a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:41.610470 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64a061-6aef-4548-a0c4-2b1f834d222a admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:41.610936 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64a061-6aef-4548-a0c4-2b1f834d222a admin admin] PATH_INFO: `/v3/users/e3b9f0f3d0214704a573f09a740f850e` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:41.612750 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64a061-6aef-4548-a0c4-2b1f834d222a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3b9f0f3d0214704a573f09a740f850e)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:41.614367 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64a061-6aef-4548-a0c4-2b1f834d222a admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:41.661407 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-cb64a061-6aef-4548-a0c4-2b1f834d222a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3b9f0f3d0214704a573f09a740f850e'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:41.669747 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-cb64a061-6aef-4548-a0c4-2b1f834d222a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3b9f0f3d0214704a573f09a740f850e'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:41.670541 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-cb64a061-6aef-4548-a0c4-2b1f834d222a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3b9f0f3d0214704a573f09a740f850e'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:41.671168 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-cb64a061-6aef-4548-a0c4-2b1f834d222a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3b9f0f3d0214704a573f09a740f850e'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:41.678460 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-cb64a061-6aef-4548-a0c4-2b1f834d222a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3b9f0f3d0214704a573f09a740f850e'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:41.695622 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1391/2803] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:16:41 2022] DELETE /identity/v3/users/e3b9f0f3d0214704a573f09a740f850e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:41.974370 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a802ab-6433-428b-bf25-c40eaa4776cc admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:41.975512 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b9a802ab-6433-428b-bf25-c40eaa4776cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:41.988757 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a802ab-6433-428b-bf25-c40eaa4776cc admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:42.049331 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9761accc-78a2-4262-9316-c8170edc628e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:42.050122 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761accc-78a2-4262-9316-c8170edc628e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:42.050466 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761accc-78a2-4262-9316-c8170edc628e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:42.050747 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761accc-78a2-4262-9316-c8170edc628e admin admin] PATH_INFO: `/v3/projects/bee4302010774fd682f6488542c50530` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:42.051966 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9761accc-78a2-4262-9316-c8170edc628e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bee4302010774fd682f6488542c50530)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:42.054389 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9761accc-78a2-4262-9316-c8170edc628e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:42.103652 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9761accc-78a2-4262-9316-c8170edc628e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bee4302010774fd682f6488542c50530', 'request_id': 'req-9761accc-78a2-4262-9316-c8170edc628e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:42.109323 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9761accc-78a2-4262-9316-c8170edc628e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bee4302010774fd682f6488542c50530', 'request_id': 'req-9761accc-78a2-4262-9316-c8170edc628e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:42.115450 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9761accc-78a2-4262-9316-c8170edc628e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bee4302010774fd682f6488542c50530', 'request_id': 'req-9761accc-78a2-4262-9316-c8170edc628e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:42.127732 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9761accc-78a2-4262-9316-c8170edc628e admin admin] The token cache is being invalidate because project bee4302010774fd682f6488542c50530 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:16:42.128018 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9761accc-78a2-4262-9316-c8170edc628e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:42.133159 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9761accc-78a2-4262-9316-c8170edc628e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:42.134791 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1413/2804] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:16:41 2022] DELETE /identity/v3/projects/bee4302010774fd682f6488542c50530 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:42.139697 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af5720-1b26-4953-9f0c-e6f5dcf3e208 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:42.140187 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af5720-1b26-4953-9f0c-e6f5dcf3e208 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:42.140300 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af5720-1b26-4953-9f0c-e6f5dcf3e208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:42.160066 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-37af5720-1b26-4953-9f0c-e6f5dcf3e208 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:42.161271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-37af5720-1b26-4953-9f0c-e6f5dcf3e208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:42.222057 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1392/2805] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:42.230184 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82501bb2-2ec3-4997-812d-ec651e68edd5 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:42.230508 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82501bb2-2ec3-4997-812d-ec651e68edd5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:42.230792 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82501bb2-2ec3-4997-812d-ec651e68edd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:42.249553 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-82501bb2-2ec3-4997-812d-ec651e68edd5 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:42.250796 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-82501bb2-2ec3-4997-812d-ec651e68edd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:42.259437 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1414/2806] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:42.263661 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af5720-1b26-4953-9f0c-e6f5dcf3e208 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:42.274456 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af5720-1b26-4953-9f0c-e6f5dcf3e208 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:42.279103 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b645d-ce5c-4b8b-83c5-ebac599cf69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:42.279761 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b645d-ce5c-4b8b-83c5-ebac599cf69b admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:42.280046 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b645d-ce5c-4b8b-83c5-ebac599cf69b admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:42.280325 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b645d-ce5c-4b8b-83c5-ebac599cf69b admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:42.281049 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b645d-ce5c-4b8b-83c5-ebac599cf69b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:16:42.281357 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b645d-ce5c-4b8b-83c5-ebac599cf69b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:42.283062 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b645d-ce5c-4b8b-83c5-ebac599cf69b admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:42.294995 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1393/2807] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 18:16:42.299276 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82501bb2-2ec3-4997-812d-ec651e68edd5 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:42.309577 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82501bb2-2ec3-4997-812d-ec651e68edd5 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:42.314059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab1a79-0cb5-45a2-9315-454d5ffaf037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:42.314745 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab1a79-0cb5-45a2-9315-454d5ffaf037 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:42.315027 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab1a79-0cb5-45a2-9315-454d5ffaf037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:42.315303 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab1a79-0cb5-45a2-9315-454d5ffaf037 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:42.316064 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ab1a79-0cb5-45a2-9315-454d5ffaf037 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:42.317957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ab1a79-0cb5-45a2-9315-454d5ffaf037 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:42.345086 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1415/2808] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:16:42 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:16:42.349180 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b645d-ce5c-4b8b-83c5-ebac599cf69b admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:42.359298 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b645d-ce5c-4b8b-83c5-ebac599cf69b admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:42.416856 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c85e3c-2f69-4ead-8a8b-4ac4c3fa5471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:42.417529 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c85e3c-2f69-4ead-8a8b-4ac4c3fa5471 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:42.418098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c85e3c-2f69-4ead-8a8b-4ac4c3fa5471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:42.418412 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c85e3c-2f69-4ead-8a8b-4ac4c3fa5471 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:42.419327 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c85e3c-2f69-4ead-8a8b-4ac4c3fa5471 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:42.422237 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c85e3c-2f69-4ead-8a8b-4ac4c3fa5471 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:42.433630 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:42.448672 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1394/2809] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:16:42 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:16:42.456029 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab1a79-0cb5-45a2-9315-454d5ffaf037 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:42.466902 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab1a79-0cb5-45a2-9315-454d5ffaf037 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:42.471627 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba1daa9-0e19-4019-97da-b220d5b6fb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:42.472293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba1daa9-0e19-4019-97da-b220d5b6fb06 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:42.472596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba1daa9-0e19-4019-97da-b220d5b6fb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:42.472956 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba1daa9-0e19-4019-97da-b220d5b6fb06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:42.473633 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba1daa9-0e19-4019-97da-b220d5b6fb06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:42.475425 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba1daa9-0e19-4019-97da-b220d5b6fb06 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:42.486098 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1416/2810] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:16:42.490001 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c85e3c-2f69-4ead-8a8b-4ac4c3fa5471 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:42.500561 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c85e3c-2f69-4ead-8a8b-4ac4c3fa5471 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:42.505180 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d6948-da9f-4f04-a4e1-28967021576a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:42.505846 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d6948-da9f-4f04-a4e1-28967021576a admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:42.507341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d6948-da9f-4f04-a4e1-28967021576a admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:42.507633 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d6948-da9f-4f04-a4e1-28967021576a admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:42.508361 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8d6948-da9f-4f04-a4e1-28967021576a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:42.509945 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8d6948-da9f-4f04-a4e1-28967021576a admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:42.519978 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1395/2811] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:16:42.524712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba1daa9-0e19-4019-97da-b220d5b6fb06 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:42.534571 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba1daa9-0e19-4019-97da-b220d5b6fb06 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:42.539969 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ac826-812b-4047-ad15-c96fb8ce2ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:42.540839 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ac826-812b-4047-ad15-c96fb8ce2ae5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:42.541124 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ac826-812b-4047-ad15-c96fb8ce2ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:42.541403 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ac826-812b-4047-ad15-c96fb8ce2ae5 admin admin] PATH_INFO: `/v3/projects/88e2443642114f7aac444b67df927809/users/e8187a0f945e442ebd716a63c1c484de/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:42.556565 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ac826-812b-4047-ad15-c96fb8ce2ae5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e2443642114f7aac444b67df927809, user_id=e8187a0f945e442ebd716a63c1c484de, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:42.559242 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ac826-812b-4047-ad15-c96fb8ce2ae5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:42.572732 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1417/2812] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:16:42 2022] PUT /identity/v3/projects/88e2443642114f7aac444b67df927809/users/e8187a0f945e442ebd716a63c1c484de/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:42.582742 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8889df0-f2f6-4e0f-804c-99445bf95802 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:42.583069 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8889df0-f2f6-4e0f-804c-99445bf95802 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:42.583324 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8889df0-f2f6-4e0f-804c-99445bf95802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:42.609678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-f8889df0-f2f6-4e0f-804c-99445bf95802 None None] MFA Rules not processed for user `e8187a0f945e442ebd716a63c1c484de`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:42.610915 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-f8889df0-f2f6-4e0f-804c-99445bf95802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:42.678300 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1396/2813] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:42.682825 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ac826-812b-4047-ad15-c96fb8ce2ae5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:42.713869 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:42.748764 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ac826-812b-4047-ad15-c96fb8ce2ae5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:42.753433 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30dd3b-011b-4b85-b7ec-f439b082dc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:42.754088 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30dd3b-011b-4b85-b7ec-f439b082dc65 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:42.754399 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30dd3b-011b-4b85-b7ec-f439b082dc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:42.754676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30dd3b-011b-4b85-b7ec-f439b082dc65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:42.755355 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30dd3b-011b-4b85-b7ec-f439b082dc65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:42.757791 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30dd3b-011b-4b85-b7ec-f439b082dc65 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:42.784245 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1418/2814] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:16:42 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:16:42.788986 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8889df0-f2f6-4e0f-804c-99445bf95802 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:42.799656 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8889df0-f2f6-4e0f-804c-99445bf95802 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:42.859288 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f260c8-8c4c-4912-93bf-85c773f570e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:42.859890 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f260c8-8c4c-4912-93bf-85c773f570e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:42.860180 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f260c8-8c4c-4912-93bf-85c773f570e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:42.860462 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f260c8-8c4c-4912-93bf-85c773f570e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:42.861157 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f260c8-8c4c-4912-93bf-85c773f570e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:42.863135 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f260c8-8c4c-4912-93bf-85c773f570e9 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:42.883144 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1397/2815] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:16:42 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:16:42.887151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30dd3b-011b-4b85-b7ec-f439b082dc65 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:42.898112 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30dd3b-011b-4b85-b7ec-f439b082dc65 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:42.903251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d686728-83d9-4bb2-b4e6-e399180d01b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:42.903922 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d686728-83d9-4bb2-b4e6-e399180d01b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:42.904211 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d686728-83d9-4bb2-b4e6-e399180d01b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:42.904492 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d686728-83d9-4bb2-b4e6-e399180d01b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:42.905215 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d686728-83d9-4bb2-b4e6-e399180d01b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:42.906869 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d686728-83d9-4bb2-b4e6-e399180d01b1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:42.917970 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1419/2816] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:16:42.922052 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f260c8-8c4c-4912-93bf-85c773f570e9 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:42.932354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f260c8-8c4c-4912-93bf-85c773f570e9 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:42.936986 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ea5df-b1cc-4f4a-85b0-ff5987072fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:42.937869 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ea5df-b1cc-4f4a-85b0-ff5987072fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:42.938151 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ea5df-b1cc-4f4a-85b0-ff5987072fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:42.938461 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ea5df-b1cc-4f4a-85b0-ff5987072fc2 admin admin] PATH_INFO: `/v3/projects/17d8ef3ca5ff48e7875d165c59a4917a/users/1ceffcc21b2a45649024e783236810da/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:42.954359 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ea5df-b1cc-4f4a-85b0-ff5987072fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17d8ef3ca5ff48e7875d165c59a4917a, user_id=1ceffcc21b2a45649024e783236810da, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:42.957006 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ea5df-b1cc-4f4a-85b0-ff5987072fc2 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:42.971374 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1398/2817] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:16:42 2022] PUT /identity/v3/projects/17d8ef3ca5ff48e7875d165c59a4917a/users/1ceffcc21b2a45649024e783236810da/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:42.976048 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d686728-83d9-4bb2-b4e6-e399180d01b1 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:42.985890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d686728-83d9-4bb2-b4e6-e399180d01b1 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:43.043363 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9e213-773e-4b13-903e-3f23e0f089c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:43.044035 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9e213-773e-4b13-903e-3f23e0f089c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:43.044322 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9e213-773e-4b13-903e-3f23e0f089c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:43.044600 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9e213-773e-4b13-903e-3f23e0f089c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:43.045330 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9e213-773e-4b13-903e-3f23e0f089c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:43.046968 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9e213-773e-4b13-903e-3f23e0f089c1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:43.057971 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1420/2818] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:16:43.061965 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ea5df-b1cc-4f4a-85b0-ff5987072fc2 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:43.072933 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ea5df-b1cc-4f4a-85b0-ff5987072fc2 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:43.077665 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7086e-7fc4-4049-a627-ae4502954782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:43.078591 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7086e-7fc4-4049-a627-ae4502954782 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:43.078878 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7086e-7fc4-4049-a627-ae4502954782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:43.079154 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7086e-7fc4-4049-a627-ae4502954782 admin admin] PATH_INFO: `/v3/projects/17d8ef3ca5ff48e7875d165c59a4917a/users/1ceffcc21b2a45649024e783236810da/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:43.081125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7086e-7fc4-4049-a627-ae4502954782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17d8ef3ca5ff48e7875d165c59a4917a, user_id=1ceffcc21b2a45649024e783236810da, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:43.083816 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7086e-7fc4-4049-a627-ae4502954782 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:43.094245 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1399/2819] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:16:43 2022] PUT /identity/v3/projects/17d8ef3ca5ff48e7875d165c59a4917a/users/1ceffcc21b2a45649024e783236810da/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:43.104498 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd186b87-5f7e-4663-b1c4-068e056d895b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:43.104797 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd186b87-5f7e-4663-b1c4-068e056d895b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:43.105084 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd186b87-5f7e-4663-b1c4-068e056d895b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:43.134010 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-dd186b87-5f7e-4663-b1c4-068e056d895b None None] MFA Rules not processed for user `1ceffcc21b2a45649024e783236810da`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:43.135169 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-dd186b87-5f7e-4663-b1c4-068e056d895b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:43.205554 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1421/2820] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:43.216868 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7086e-7fc4-4049-a627-ae4502954782 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:43.218687 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-17f7086e-7fc4-4049-a627-ae4502954782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:43.248565 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:43.279952 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7086e-7fc4-4049-a627-ae4502954782 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:43.283607 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1cbd7b-0af5-4ba3-a7c9-354f4b7fcacc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:43.284259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1cbd7b-0af5-4ba3-a7c9-354f4b7fcacc service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:43.284545 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1cbd7b-0af5-4ba3-a7c9-354f4b7fcacc service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:43.284826 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1cbd7b-0af5-4ba3-a7c9-354f4b7fcacc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:43.293975 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1cbd7b-0af5-4ba3-a7c9-354f4b7fcacc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:43.295443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1cbd7b-0af5-4ba3-a7c9-354f4b7fcacc service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:43.305624 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1400/2821] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:16:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:16:43.333718 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd186b87-5f7e-4663-b1c4-068e056d895b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:43.359793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:43.399154 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd186b87-5f7e-4663-b1c4-068e056d895b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:43.403718 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a9ba2-71cc-4267-aedd-356fd69ca011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:43.404599 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a9ba2-71cc-4267-aedd-356fd69ca011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:43.404880 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a9ba2-71cc-4267-aedd-356fd69ca011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:43.405158 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a9ba2-71cc-4267-aedd-356fd69ca011 admin admin] PATH_INFO: `/v3/users/e8187a0f945e442ebd716a63c1c484de` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:43.406378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a9ba2-71cc-4267-aedd-356fd69ca011 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8187a0f945e442ebd716a63c1c484de)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:43.408293 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a9ba2-71cc-4267-aedd-356fd69ca011 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:43.454346 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-540a9ba2-71cc-4267-aedd-356fd69ca011 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8187a0f945e442ebd716a63c1c484de'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:43.461937 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-540a9ba2-71cc-4267-aedd-356fd69ca011 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8187a0f945e442ebd716a63c1c484de'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:43.473883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-540a9ba2-71cc-4267-aedd-356fd69ca011 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8187a0f945e442ebd716a63c1c484de'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:43.475025 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-540a9ba2-71cc-4267-aedd-356fd69ca011 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8187a0f945e442ebd716a63c1c484de'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:43.475716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-540a9ba2-71cc-4267-aedd-356fd69ca011 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8187a0f945e442ebd716a63c1c484de'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:43.487908 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1422/2822] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:16:43 2022] DELETE /identity/v3/users/e8187a0f945e442ebd716a63c1c484de => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:43.493056 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f98dce-40c9-4656-9d17-43426a22237c None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:43.493350 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f98dce-40c9-4656-9d17-43426a22237c None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:43.493634 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f98dce-40c9-4656-9d17-43426a22237c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:43.511429 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-53f98dce-40c9-4656-9d17-43426a22237c None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:43.512633 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-53f98dce-40c9-4656-9d17-43426a22237c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:43.575557 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1401/2823] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:43.585712 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a9ba2-71cc-4267-aedd-356fd69ca011 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:43.587152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-540a9ba2-71cc-4267-aedd-356fd69ca011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:43.645331 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a9ba2-71cc-4267-aedd-356fd69ca011 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:43.649960 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa0ee7-af3d-4750-badc-444c0d2b1916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:43.650572 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa0ee7-af3d-4750-badc-444c0d2b1916 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:43.650861 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa0ee7-af3d-4750-badc-444c0d2b1916 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:43.651131 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa0ee7-af3d-4750-badc-444c0d2b1916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:43.660180 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa0ee7-af3d-4750-badc-444c0d2b1916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:43.661606 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa0ee7-af3d-4750-badc-444c0d2b1916 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:43.672414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1423/2824] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:16:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:16:43.953942 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f98dce-40c9-4656-9d17-43426a22237c None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:43.955499 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-53f98dce-40c9-4656-9d17-43426a22237c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:43.969909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f98dce-40c9-4656-9d17-43426a22237c None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:43.975066 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba080ea-9ba4-48e9-8d5e-923c369b0fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:43.976020 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba080ea-9ba4-48e9-8d5e-923c369b0fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:43.976308 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba080ea-9ba4-48e9-8d5e-923c369b0fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:43.976586 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba080ea-9ba4-48e9-8d5e-923c369b0fce admin admin] PATH_INFO: `/v3/projects/88e2443642114f7aac444b67df927809` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:43.977798 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba080ea-9ba4-48e9-8d5e-923c369b0fce admin admin] RBAC: Authorizing `identity:delete_project(project_id=88e2443642114f7aac444b67df927809)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:43.979773 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba080ea-9ba4-48e9-8d5e-923c369b0fce admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:44.028978 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-fba080ea-9ba4-48e9-8d5e-923c369b0fce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88e2443642114f7aac444b67df927809', 'request_id': 'req-fba080ea-9ba4-48e9-8d5e-923c369b0fce'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:44.036280 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-fba080ea-9ba4-48e9-8d5e-923c369b0fce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88e2443642114f7aac444b67df927809', 'request_id': 'req-fba080ea-9ba4-48e9-8d5e-923c369b0fce'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:44.042522 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-fba080ea-9ba4-48e9-8d5e-923c369b0fce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88e2443642114f7aac444b67df927809', 'request_id': 'req-fba080ea-9ba4-48e9-8d5e-923c369b0fce'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:44.055253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-fba080ea-9ba4-48e9-8d5e-923c369b0fce admin admin] The token cache is being invalidate because project 88e2443642114f7aac444b67df927809 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:16:44.055519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-fba080ea-9ba4-48e9-8d5e-923c369b0fce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:44.056169 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-fba080ea-9ba4-48e9-8d5e-923c369b0fce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:44.057738 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1402/2825] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:16:43 2022] DELETE /identity/v3/projects/88e2443642114f7aac444b67df927809 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:16:44.061696 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa0ee7-af3d-4750-badc-444c0d2b1916 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:44.072202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa0ee7-af3d-4750-badc-444c0d2b1916 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:44.113326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:44.130526 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839dad2e-bc74-487d-8dcd-f8cf8f1f1976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:44.132048 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839dad2e-bc74-487d-8dcd-f8cf8f1f1976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:44.132332 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839dad2e-bc74-487d-8dcd-f8cf8f1f1976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:44.132610 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839dad2e-bc74-487d-8dcd-f8cf8f1f1976 admin admin] PATH_INFO: `/v3/users/1ceffcc21b2a45649024e783236810da` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:44.133830 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839dad2e-bc74-487d-8dcd-f8cf8f1f1976 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ceffcc21b2a45649024e783236810da)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:44.135793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839dad2e-bc74-487d-8dcd-f8cf8f1f1976 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:44.182322 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-839dad2e-bc74-487d-8dcd-f8cf8f1f1976 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ceffcc21b2a45649024e783236810da'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:44.189423 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-839dad2e-bc74-487d-8dcd-f8cf8f1f1976 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ceffcc21b2a45649024e783236810da'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:44.201889 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-839dad2e-bc74-487d-8dcd-f8cf8f1f1976 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ceffcc21b2a45649024e783236810da'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:44.202533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-839dad2e-bc74-487d-8dcd-f8cf8f1f1976 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ceffcc21b2a45649024e783236810da'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:44.203208 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-839dad2e-bc74-487d-8dcd-f8cf8f1f1976 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ceffcc21b2a45649024e783236810da'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:44.215435 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1424/2826] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:16:44 2022] DELETE /identity/v3/users/1ceffcc21b2a45649024e783236810da => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:16:44.495405 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba080ea-9ba4-48e9-8d5e-923c369b0fce admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:44.496958 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-fba080ea-9ba4-48e9-8d5e-923c369b0fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:44.513400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba080ea-9ba4-48e9-8d5e-923c369b0fce admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:44.564891 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:44.576956 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbfba2-b0e2-40de-a4ce-ad26586aa971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:44.577963 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbfba2-b0e2-40de-a4ce-ad26586aa971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:44.578347 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbfba2-b0e2-40de-a4ce-ad26586aa971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:44.578898 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbfba2-b0e2-40de-a4ce-ad26586aa971 admin admin] PATH_INFO: `/v3/projects/17d8ef3ca5ff48e7875d165c59a4917a` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:44.580098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbfba2-b0e2-40de-a4ce-ad26586aa971 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17d8ef3ca5ff48e7875d165c59a4917a)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:44.582033 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbfba2-b0e2-40de-a4ce-ad26586aa971 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:44.628339 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-50cbfba2-b0e2-40de-a4ce-ad26586aa971 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17d8ef3ca5ff48e7875d165c59a4917a', 'request_id': 'req-50cbfba2-b0e2-40de-a4ce-ad26586aa971'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:44.636174 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-50cbfba2-b0e2-40de-a4ce-ad26586aa971 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17d8ef3ca5ff48e7875d165c59a4917a', 'request_id': 'req-50cbfba2-b0e2-40de-a4ce-ad26586aa971'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:44.641901 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-50cbfba2-b0e2-40de-a4ce-ad26586aa971 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17d8ef3ca5ff48e7875d165c59a4917a', 'request_id': 'req-50cbfba2-b0e2-40de-a4ce-ad26586aa971'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:44.653457 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-50cbfba2-b0e2-40de-a4ce-ad26586aa971 admin admin] The token cache is being invalidate because project 17d8ef3ca5ff48e7875d165c59a4917a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:16:44.653749 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-50cbfba2-b0e2-40de-a4ce-ad26586aa971 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:44.654494 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-50cbfba2-b0e2-40de-a4ce-ad26586aa971 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:44.656026 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1403/2827] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:16:44 2022] DELETE /identity/v3/projects/17d8ef3ca5ff48e7875d165c59a4917a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:44.661126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08db74-f612-4cca-abe4-511565a9dd2b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:44.661421 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08db74-f612-4cca-abe4-511565a9dd2b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:44.661707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08db74-f612-4cca-abe4-511565a9dd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:44.692440 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4f08db74-f612-4cca-abe4-511565a9dd2b None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:44.694339 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4f08db74-f612-4cca-abe4-511565a9dd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:44.753261 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1425/2828] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:44.762809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73553e-9801-4001-9115-a4fdc22a79c7 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:44.763097 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73553e-9801-4001-9115-a4fdc22a79c7 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:44.763504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73553e-9801-4001-9115-a4fdc22a79c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:44.783060 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-3d73553e-9801-4001-9115-a4fdc22a79c7 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:44.784272 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-3d73553e-9801-4001-9115-a4fdc22a79c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:44.792584 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1404/2829] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:44.796843 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08db74-f612-4cca-abe4-511565a9dd2b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:44.809065 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08db74-f612-4cca-abe4-511565a9dd2b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:44.813678 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ac44e-844d-4a2c-867f-1b4c2c2db666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:44.814407 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ac44e-844d-4a2c-867f-1b4c2c2db666 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:44.814713 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ac44e-844d-4a2c-867f-1b4c2c2db666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:44.814980 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ac44e-844d-4a2c-867f-1b4c2c2db666 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:44.815708 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ac44e-844d-4a2c-867f-1b4c2c2db666 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:16:44.816017 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ac44e-844d-4a2c-867f-1b4c2c2db666 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:44.817629 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ac44e-844d-4a2c-867f-1b4c2c2db666 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:44.830523 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1426/2830] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:16:44.835493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73553e-9801-4001-9115-a4fdc22a79c7 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:44.845844 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73553e-9801-4001-9115-a4fdc22a79c7 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:44.851182 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592b0a5-5040-437d-bfda-b5a0447570f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:44.851844 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592b0a5-5040-437d-bfda-b5a0447570f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:44.852130 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592b0a5-5040-437d-bfda-b5a0447570f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:44.852406 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592b0a5-5040-437d-bfda-b5a0447570f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:44.853111 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6592b0a5-5040-437d-bfda-b5a0447570f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:44.855111 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6592b0a5-5040-437d-bfda-b5a0447570f3 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:44.882431 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1405/2831] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:16:44 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:16:44.886751 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ac44e-844d-4a2c-867f-1b4c2c2db666 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:44.900006 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ac44e-844d-4a2c-867f-1b4c2c2db666 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:44.940974 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:44.960656 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84a065-7045-43b8-9efa-c2b9d10780bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:44.961329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84a065-7045-43b8-9efa-c2b9d10780bf admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:44.961619 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84a065-7045-43b8-9efa-c2b9d10780bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:44.961901 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84a065-7045-43b8-9efa-c2b9d10780bf admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:44.962620 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a84a065-7045-43b8-9efa-c2b9d10780bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:44.964588 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a84a065-7045-43b8-9efa-c2b9d10780bf admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:44.986941 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1427/2832] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:16:44 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:16:44.991699 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592b0a5-5040-437d-bfda-b5a0447570f3 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:45.002266 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592b0a5-5040-437d-bfda-b5a0447570f3 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:45.007035 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e8ec1-e0be-441a-adbd-0b1af6e31398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:45.007667 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e8ec1-e0be-441a-adbd-0b1af6e31398 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:45.007957 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e8ec1-e0be-441a-adbd-0b1af6e31398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:45.008237 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e8ec1-e0be-441a-adbd-0b1af6e31398 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:45.008964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e8ec1-e0be-441a-adbd-0b1af6e31398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:45.010611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e8ec1-e0be-441a-adbd-0b1af6e31398 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:45.021733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1406/2833] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:16:45.026156 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84a065-7045-43b8-9efa-c2b9d10780bf admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:45.036576 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84a065-7045-43b8-9efa-c2b9d10780bf admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:45.041716 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ecdf4-d153-4513-a777-bd0fec4a9992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:45.042405 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ecdf4-d153-4513-a777-bd0fec4a9992 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:45.042690 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ecdf4-d153-4513-a777-bd0fec4a9992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:45.042965 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ecdf4-d153-4513-a777-bd0fec4a9992 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:45.043715 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ecdf4-d153-4513-a777-bd0fec4a9992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:45.045337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ecdf4-d153-4513-a777-bd0fec4a9992 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:45.056739 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1428/2834] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:16:45.060909 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e8ec1-e0be-441a-adbd-0b1af6e31398 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:45.070888 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e8ec1-e0be-441a-adbd-0b1af6e31398 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:45.075401 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c8ff6-fa7a-4ab6-8897-f6cf69dac8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:45.076278 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c8ff6-fa7a-4ab6-8897-f6cf69dac8a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:45.076569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c8ff6-fa7a-4ab6-8897-f6cf69dac8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:45.076847 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c8ff6-fa7a-4ab6-8897-f6cf69dac8a8 admin admin] PATH_INFO: `/v3/projects/5be68ea7d2e349d1bed46a6f58516521/users/1f2b5fd5503f454a949dc02aaf71d815/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:45.091440 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c8ff6-fa7a-4ab6-8897-f6cf69dac8a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5be68ea7d2e349d1bed46a6f58516521, user_id=1f2b5fd5503f454a949dc02aaf71d815, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:45.094090 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c8ff6-fa7a-4ab6-8897-f6cf69dac8a8 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:45.107550 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1407/2835] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:16:45 2022] PUT /identity/v3/projects/5be68ea7d2e349d1bed46a6f58516521/users/1f2b5fd5503f454a949dc02aaf71d815/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:45.119566 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437fa50-171c-4b3f-86d5-5eea9ef35d94 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:45.120082 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437fa50-171c-4b3f-86d5-5eea9ef35d94 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:45.120082 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437fa50-171c-4b3f-86d5-5eea9ef35d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:45.147155 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a437fa50-171c-4b3f-86d5-5eea9ef35d94 None None] MFA Rules not processed for user `1f2b5fd5503f454a949dc02aaf71d815`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:45.148212 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a437fa50-171c-4b3f-86d5-5eea9ef35d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:45.214510 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1429/2836] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:45.226787 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c8ff6-fa7a-4ab6-8897-f6cf69dac8a8 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:45.228447 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-b64c8ff6-fa7a-4ab6-8897-f6cf69dac8a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:45.285722 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c8ff6-fa7a-4ab6-8897-f6cf69dac8a8 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:45.290162 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf73af2-d3e9-48a9-9c52-7db431170e18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:45.290832 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf73af2-d3e9-48a9-9c52-7db431170e18 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:45.291117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf73af2-d3e9-48a9-9c52-7db431170e18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:45.291393 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf73af2-d3e9-48a9-9c52-7db431170e18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:45.299745 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf73af2-d3e9-48a9-9c52-7db431170e18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:45.301271 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf73af2-d3e9-48a9-9c52-7db431170e18 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:45.311277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1408/2837] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:16:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 18:16:45.330689 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a437fa50-171c-4b3f-86d5-5eea9ef35d94 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:45.392894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a437fa50-171c-4b3f-86d5-5eea9ef35d94 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:45.397989 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef20b6-a52a-4875-8359-ab867dace75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:45.399152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef20b6-a52a-4875-8359-ab867dace75d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:45.399431 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef20b6-a52a-4875-8359-ab867dace75d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:45.399703 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef20b6-a52a-4875-8359-ab867dace75d admin admin] PATH_INFO: `/v3/users/1f2b5fd5503f454a949dc02aaf71d815` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:45.400890 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef20b6-a52a-4875-8359-ab867dace75d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f2b5fd5503f454a949dc02aaf71d815)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:45.402861 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef20b6-a52a-4875-8359-ab867dace75d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:45.449520 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-79ef20b6-a52a-4875-8359-ab867dace75d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f2b5fd5503f454a949dc02aaf71d815'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:45.453196 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:45.457467 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-79ef20b6-a52a-4875-8359-ab867dace75d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f2b5fd5503f454a949dc02aaf71d815'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:45.470836 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-79ef20b6-a52a-4875-8359-ab867dace75d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f2b5fd5503f454a949dc02aaf71d815'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:45.471504 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-79ef20b6-a52a-4875-8359-ab867dace75d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f2b5fd5503f454a949dc02aaf71d815'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:45.472204 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-79ef20b6-a52a-4875-8359-ab867dace75d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f2b5fd5503f454a949dc02aaf71d815'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:45.484859 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1430/2838] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:16:45 2022] DELETE /identity/v3/users/1f2b5fd5503f454a949dc02aaf71d815 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:45.489582 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d61c4-0228-4df4-ac5d-be5ac6abd8fb None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:45.489944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d61c4-0228-4df4-ac5d-be5ac6abd8fb None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:45.490758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d61c4-0228-4df4-ac5d-be5ac6abd8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:45.508235 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-086d61c4-0228-4df4-ac5d-be5ac6abd8fb None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:45.509427 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-086d61c4-0228-4df4-ac5d-be5ac6abd8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:45.522796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:45.572903 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1409/2839] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:45.583148 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef20b6-a52a-4875-8359-ab867dace75d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:45.584668 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-79ef20b6-a52a-4875-8359-ab867dace75d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:45.646266 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef20b6-a52a-4875-8359-ab867dace75d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:45.650744 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64db9a7-3969-4002-8b46-48160ba9f37b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:45.651517 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64db9a7-3969-4002-8b46-48160ba9f37b service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:45.651800 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64db9a7-3969-4002-8b46-48160ba9f37b service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:45.652092 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64db9a7-3969-4002-8b46-48160ba9f37b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:45.661432 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64db9a7-3969-4002-8b46-48160ba9f37b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:45.662953 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64db9a7-3969-4002-8b46-48160ba9f37b service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:45.672317 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1431/2840] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:16:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:16:45.945782 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d61c4-0228-4df4-ac5d-be5ac6abd8fb None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:45.947220 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-086d61c4-0228-4df4-ac5d-be5ac6abd8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:45.962544 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d61c4-0228-4df4-ac5d-be5ac6abd8fb None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:45.968388 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77941332-676d-4673-943e-c184a38112e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:45.969642 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77941332-676d-4673-943e-c184a38112e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:45.969925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77941332-676d-4673-943e-c184a38112e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:45.970240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77941332-676d-4673-943e-c184a38112e6 admin admin] PATH_INFO: `/v3/projects/5be68ea7d2e349d1bed46a6f58516521` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:45.971671 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77941332-676d-4673-943e-c184a38112e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5be68ea7d2e349d1bed46a6f58516521)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:45.973719 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77941332-676d-4673-943e-c184a38112e6 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:46.019905 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-77941332-676d-4673-943e-c184a38112e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5be68ea7d2e349d1bed46a6f58516521', 'request_id': 'req-77941332-676d-4673-943e-c184a38112e6'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:46.028173 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-77941332-676d-4673-943e-c184a38112e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5be68ea7d2e349d1bed46a6f58516521', 'request_id': 'req-77941332-676d-4673-943e-c184a38112e6'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:46.033925 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-77941332-676d-4673-943e-c184a38112e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5be68ea7d2e349d1bed46a6f58516521', 'request_id': 'req-77941332-676d-4673-943e-c184a38112e6'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:46.045240 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-77941332-676d-4673-943e-c184a38112e6 admin admin] The token cache is being invalidate because project 5be68ea7d2e349d1bed46a6f58516521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80260) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:16:46.045532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-77941332-676d-4673-943e-c184a38112e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:46.046171 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-77941332-676d-4673-943e-c184a38112e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:46.047756 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1410/2841] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:16:45 2022] DELETE /identity/v3/projects/5be68ea7d2e349d1bed46a6f58516521 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:46.053660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e70dbe-68a5-49f0-9bfb-602570c450d9 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:46.053954 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e70dbe-68a5-49f0-9bfb-602570c450d9 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:46.054298 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e70dbe-68a5-49f0-9bfb-602570c450d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:46.072894 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-08e70dbe-68a5-49f0-9bfb-602570c450d9 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:46.074096 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-08e70dbe-68a5-49f0-9bfb-602570c450d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:46.134793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1432/2842] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:46.142448 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e726df-9b72-4920-a6c5-6d3191d94226 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:46.142916 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e726df-9b72-4920-a6c5-6d3191d94226 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:46.143293 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e726df-9b72-4920-a6c5-6d3191d94226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:46.165215 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-43e726df-9b72-4920-a6c5-6d3191d94226 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:46.166457 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-43e726df-9b72-4920-a6c5-6d3191d94226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:46.174239 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1411/2843] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:46.178494 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e70dbe-68a5-49f0-9bfb-602570c450d9 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:46.188255 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e70dbe-68a5-49f0-9bfb-602570c450d9 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:46.192943 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fbca3b-72f7-496f-b5ab-c608c4284643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:46.194092 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbca3b-72f7-496f-b5ab-c608c4284643 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:46.194408 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbca3b-72f7-496f-b5ab-c608c4284643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:46.194687 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbca3b-72f7-496f-b5ab-c608c4284643 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:46.195403 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fbca3b-72f7-496f-b5ab-c608c4284643 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:16:46.195707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fbca3b-72f7-496f-b5ab-c608c4284643 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:46.197312 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fbca3b-72f7-496f-b5ab-c608c4284643 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:46.208702 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1433/2844] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:16:46.213611 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e726df-9b72-4920-a6c5-6d3191d94226 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:46.222839 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e726df-9b72-4920-a6c5-6d3191d94226 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:46.228594 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337c057-c5e1-424c-95c6-c6ff8a120ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:46.229244 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337c057-c5e1-424c-95c6-c6ff8a120ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:46.229525 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337c057-c5e1-424c-95c6-c6ff8a120ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:46.229799 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337c057-c5e1-424c-95c6-c6ff8a120ae4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:46.230532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337c057-c5e1-424c-95c6-c6ff8a120ae4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:46.232421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337c057-c5e1-424c-95c6-c6ff8a120ae4 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:46.260108 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1412/2845] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:16:46 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:16:46.264219 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fbca3b-72f7-496f-b5ab-c608c4284643 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:46.275491 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fbca3b-72f7-496f-b5ab-c608c4284643 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:46.333074 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1787c3d-fd14-4d64-9a7f-9f68440859ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:46.333752 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1787c3d-fd14-4d64-9a7f-9f68440859ab admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:46.334734 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1787c3d-fd14-4d64-9a7f-9f68440859ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:46.335066 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1787c3d-fd14-4d64-9a7f-9f68440859ab admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:46.335750 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1787c3d-fd14-4d64-9a7f-9f68440859ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:46.337674 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1787c3d-fd14-4d64-9a7f-9f68440859ab admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:46.344452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:46.360019 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1434/2846] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:16:46 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:16:46.364329 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337c057-c5e1-424c-95c6-c6ff8a120ae4 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:46.375035 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337c057-c5e1-424c-95c6-c6ff8a120ae4 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:46.379651 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea417f-6922-403d-90cc-6424e95c4319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:46.380335 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea417f-6922-403d-90cc-6424e95c4319 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:46.380615 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea417f-6922-403d-90cc-6424e95c4319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:46.381048 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea417f-6922-403d-90cc-6424e95c4319 admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:46.381968 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea417f-6922-403d-90cc-6424e95c4319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:46.383602 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea417f-6922-403d-90cc-6424e95c4319 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:46.393914 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1413/2847] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:16:46.398366 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1787c3d-fd14-4d64-9a7f-9f68440859ab admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:46.408585 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1787c3d-fd14-4d64-9a7f-9f68440859ab admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:46.414452 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33abda-a0b9-45ed-a6bf-8de84636ec7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:46.415123 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33abda-a0b9-45ed-a6bf-8de84636ec7d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:46.415409 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33abda-a0b9-45ed-a6bf-8de84636ec7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:46.415688 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33abda-a0b9-45ed-a6bf-8de84636ec7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:46.416414 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33abda-a0b9-45ed-a6bf-8de84636ec7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:46.418036 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33abda-a0b9-45ed-a6bf-8de84636ec7d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:46.428181 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1435/2848] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:16:46.432380 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea417f-6922-403d-90cc-6424e95c4319 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:46.442155 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea417f-6922-403d-90cc-6424e95c4319 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:46.447800 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37f52a-d2b4-4199-9f7e-f5039bb524b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:46.448679 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37f52a-d2b4-4199-9f7e-f5039bb524b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:46.448961 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37f52a-d2b4-4199-9f7e-f5039bb524b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:46.449234 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37f52a-d2b4-4199-9f7e-f5039bb524b2 admin admin] PATH_INFO: `/v3/projects/4db395eeabf54bfba70236de291c2044/users/517c37ed55804482b3914651af9b0af4/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:46.464665 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37f52a-d2b4-4199-9f7e-f5039bb524b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4db395eeabf54bfba70236de291c2044, user_id=517c37ed55804482b3914651af9b0af4, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:46.467337 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37f52a-d2b4-4199-9f7e-f5039bb524b2 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:46.482088 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1414/2849] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:16:46 2022] PUT /identity/v3/projects/4db395eeabf54bfba70236de291c2044/users/517c37ed55804482b3914651af9b0af4/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:46.493223 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1365256-f394-41db-b3b0-4d2b7a4250a1 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:46.493547 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1365256-f394-41db-b3b0-4d2b7a4250a1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:46.493831 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1365256-f394-41db-b3b0-4d2b7a4250a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:46.521242 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b1365256-f394-41db-b3b0-4d2b7a4250a1 None None] MFA Rules not processed for user `517c37ed55804482b3914651af9b0af4`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:46.523865 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b1365256-f394-41db-b3b0-4d2b7a4250a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:46.589211 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1436/2850] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:46.593853 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37f52a-d2b4-4199-9f7e-f5039bb524b2 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:46.637523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:46.660330 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37f52a-d2b4-4199-9f7e-f5039bb524b2 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:46.665758 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1271d-e150-4729-b2e0-1c684b7913af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:46.666524 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1271d-e150-4729-b2e0-1c684b7913af admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:46.666809 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1271d-e150-4729-b2e0-1c684b7913af admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:46.667085 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1271d-e150-4729-b2e0-1c684b7913af admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:46.667784 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1271d-e150-4729-b2e0-1c684b7913af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:46.669734 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1271d-e150-4729-b2e0-1c684b7913af admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:46.695372 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1415/2851] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:16:46 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:16:46.699658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1365256-f394-41db-b3b0-4d2b7a4250a1 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:46.711167 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1365256-f394-41db-b3b0-4d2b7a4250a1 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:46.767360 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554d103-3926-48b3-ae4d-b07eef09b157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:46.768041 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554d103-3926-48b3-ae4d-b07eef09b157 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:46.768327 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554d103-3926-48b3-ae4d-b07eef09b157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:46.768608 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554d103-3926-48b3-ae4d-b07eef09b157 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:46.769314 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554d103-3926-48b3-ae4d-b07eef09b157 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:46.771294 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554d103-3926-48b3-ae4d-b07eef09b157 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:46.793682 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1437/2852] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:16:46 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:16:46.797669 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1271d-e150-4729-b2e0-1c684b7913af admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:46.808739 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1271d-e150-4729-b2e0-1c684b7913af admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:46.813438 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c54e3-b3fa-464f-b6e7-443f0a071d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:46.814103 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c54e3-b3fa-464f-b6e7-443f0a071d5a admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:46.814422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c54e3-b3fa-464f-b6e7-443f0a071d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:46.814702 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c54e3-b3fa-464f-b6e7-443f0a071d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:46.815551 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c54e3-b3fa-464f-b6e7-443f0a071d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:46.817151 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c54e3-b3fa-464f-b6e7-443f0a071d5a admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:46.828997 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1416/2853] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:16:46.833192 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554d103-3926-48b3-ae4d-b07eef09b157 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:46.843507 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554d103-3926-48b3-ae4d-b07eef09b157 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:46.848312 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f056a6a-3f76-46d8-be87-710963e812f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:46.848973 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f056a6a-3f76-46d8-be87-710963e812f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:46.849262 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f056a6a-3f76-46d8-be87-710963e812f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:46.849557 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f056a6a-3f76-46d8-be87-710963e812f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:46.850501 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f056a6a-3f76-46d8-be87-710963e812f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:46.852113 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f056a6a-3f76-46d8-be87-710963e812f3 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:46.862126 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1438/2854] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:16:46.867420 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c54e3-b3fa-464f-b6e7-443f0a071d5a admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:46.878962 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c54e3-b3fa-464f-b6e7-443f0a071d5a admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:46.884249 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656b1a7-4999-4ee0-ae61-e26f9f07a530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:46.885129 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656b1a7-4999-4ee0-ae61-e26f9f07a530 admin admin] REQUEST_METHOD: `PUT` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:46.885415 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656b1a7-4999-4ee0-ae61-e26f9f07a530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:46.885694 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656b1a7-4999-4ee0-ae61-e26f9f07a530 admin admin] PATH_INFO: `/v3/projects/81e71a0f586a46d0b5239023599e6de4/users/9b44db3704ad49cf93c433b16cfbffc6/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:46.900796 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5656b1a7-4999-4ee0-ae61-e26f9f07a530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81e71a0f586a46d0b5239023599e6de4, user_id=9b44db3704ad49cf93c433b16cfbffc6, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:46.903518 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5656b1a7-4999-4ee0-ae61-e26f9f07a530 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:46.916296 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1417/2855] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:16:46 2022] PUT /identity/v3/projects/81e71a0f586a46d0b5239023599e6de4/users/9b44db3704ad49cf93c433b16cfbffc6/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:46.926462 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46ec1e-9925-4370-8fae-7da14ac3066c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:46.926760 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46ec1e-9925-4370-8fae-7da14ac3066c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:46.927058 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46ec1e-9925-4370-8fae-7da14ac3066c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:46.955073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ff46ec1e-9925-4370-8fae-7da14ac3066c None None] MFA Rules not processed for user `9b44db3704ad49cf93c433b16cfbffc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:46.956436 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ff46ec1e-9925-4370-8fae-7da14ac3066c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:46.993303 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:47.026347 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1439/2856] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:16:47.030874 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656b1a7-4999-4ee0-ae61-e26f9f07a530 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:47.092549 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656b1a7-4999-4ee0-ae61-e26f9f07a530 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:47.097238 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa49d96-170e-42ab-9690-8ba8c2b14c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:47.097901 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa49d96-170e-42ab-9690-8ba8c2b14c32 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:47.098217 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa49d96-170e-42ab-9690-8ba8c2b14c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:47.098498 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa49d96-170e-42ab-9690-8ba8c2b14c32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:47.099205 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa49d96-170e-42ab-9690-8ba8c2b14c32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:47.101117 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa49d96-170e-42ab-9690-8ba8c2b14c32 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:47.126841 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1418/2857] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:16:47 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:16:47.131710 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46ec1e-9925-4370-8fae-7da14ac3066c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:47.142790 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46ec1e-9925-4370-8fae-7da14ac3066c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:47.202181 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd3675-6dbf-4a95-b92a-78e578e86f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:47.202863 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd3675-6dbf-4a95-b92a-78e578e86f21 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:47.203080 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd3675-6dbf-4a95-b92a-78e578e86f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:47.203357 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd3675-6dbf-4a95-b92a-78e578e86f21 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:47.204276 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd3675-6dbf-4a95-b92a-78e578e86f21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:47.205971 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd3675-6dbf-4a95-b92a-78e578e86f21 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:47.226689 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1440/2858] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:16:47 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:16:47.230905 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa49d96-170e-42ab-9690-8ba8c2b14c32 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:47.243866 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa49d96-170e-42ab-9690-8ba8c2b14c32 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:47.248553 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce8029-7ae2-491b-99cd-1bdd5a3f3afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:47.249217 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce8029-7ae2-491b-99cd-1bdd5a3f3afd admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:47.249504 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce8029-7ae2-491b-99cd-1bdd5a3f3afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:47.249814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce8029-7ae2-491b-99cd-1bdd5a3f3afd admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:47.250569 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce8029-7ae2-491b-99cd-1bdd5a3f3afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:47.252201 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce8029-7ae2-491b-99cd-1bdd5a3f3afd admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:47.262982 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1419/2859] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:16:47.267154 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd3675-6dbf-4a95-b92a-78e578e86f21 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:47.278128 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd3675-6dbf-4a95-b92a-78e578e86f21 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:47.286361 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fd0e3-4564-4a50-895b-3588208bccd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:47.287845 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fd0e3-4564-4a50-895b-3588208bccd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:47.288313 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fd0e3-4564-4a50-895b-3588208bccd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:47.288596 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fd0e3-4564-4a50-895b-3588208bccd9 admin admin] PATH_INFO: `/v3/projects/8ffe788b6eeb49d6a9eee4d7378e119f/users/063465380cb94adcacb15d68ced98ff9/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:47.305107 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1fd0e3-4564-4a50-895b-3588208bccd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ffe788b6eeb49d6a9eee4d7378e119f, user_id=063465380cb94adcacb15d68ced98ff9, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:47.307848 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1fd0e3-4564-4a50-895b-3588208bccd9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:47.320950 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1441/2860] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:16:47 2022] PUT /identity/v3/projects/8ffe788b6eeb49d6a9eee4d7378e119f/users/063465380cb94adcacb15d68ced98ff9/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:47.325078 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce8029-7ae2-491b-99cd-1bdd5a3f3afd admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:47.335315 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce8029-7ae2-491b-99cd-1bdd5a3f3afd admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:47.392340 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d0cd9-dbc5-4ed3-bc53-a22775a87d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:47.393008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d0cd9-dbc5-4ed3-bc53-a22775a87d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:47.393295 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d0cd9-dbc5-4ed3-bc53-a22775a87d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:47.393576 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d0cd9-dbc5-4ed3-bc53-a22775a87d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:47.394331 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2d0cd9-dbc5-4ed3-bc53-a22775a87d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:47.395935 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2d0cd9-dbc5-4ed3-bc53-a22775a87d4d admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:47.401408 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:47.409863 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1420/2861] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 18:16:47.411326 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fd0e3-4564-4a50-895b-3588208bccd9 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:47.422906 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fd0e3-4564-4a50-895b-3588208bccd9 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:47.427393 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efddee87-8f4a-436c-adda-403ded43456c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:47.428921 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efddee87-8f4a-436c-adda-403ded43456c admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:47.429200 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efddee87-8f4a-436c-adda-403ded43456c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:47.429474 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efddee87-8f4a-436c-adda-403ded43456c admin admin] PATH_INFO: `/v3/projects/8ffe788b6eeb49d6a9eee4d7378e119f/users/063465380cb94adcacb15d68ced98ff9/roles/4fa8f06085c741a5b3cf64e63cc0754f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:47.431449 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efddee87-8f4a-436c-adda-403ded43456c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ffe788b6eeb49d6a9eee4d7378e119f, user_id=063465380cb94adcacb15d68ced98ff9, role_id=4fa8f06085c741a5b3cf64e63cc0754f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:47.434080 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efddee87-8f4a-436c-adda-403ded43456c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:47.444403 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1442/2862] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:16:47 2022] PUT /identity/v3/projects/8ffe788b6eeb49d6a9eee4d7378e119f/users/063465380cb94adcacb15d68ced98ff9/roles/4fa8f06085c741a5b3cf64e63cc0754f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:47.454899 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e6bec-74d6-49ed-9897-5a03d148a482 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:47.455137 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e6bec-74d6-49ed-9897-5a03d148a482 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:47.455420 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e6bec-74d6-49ed-9897-5a03d148a482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:47.482971 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-264e6bec-74d6-49ed-9897-5a03d148a482 None None] MFA Rules not processed for user `063465380cb94adcacb15d68ced98ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:47.484214 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-264e6bec-74d6-49ed-9897-5a03d148a482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:47.555028 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1421/2863] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:47.566177 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efddee87-8f4a-436c-adda-403ded43456c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:47.567930 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-efddee87-8f4a-436c-adda-403ded43456c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:47.627163 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efddee87-8f4a-436c-adda-403ded43456c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:47.631252 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46dc07f-1431-4d50-b311-302bf6e545d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:47.631987 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46dc07f-1431-4d50-b311-302bf6e545d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:47.632276 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46dc07f-1431-4d50-b311-302bf6e545d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:47.632555 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46dc07f-1431-4d50-b311-302bf6e545d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:47.675509 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:47.691441 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46dc07f-1431-4d50-b311-302bf6e545d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:47.692901 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46dc07f-1431-4d50-b311-302bf6e545d3 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:47.701914 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1443/2864] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:16:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:16:49.194662 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e6bec-74d6-49ed-9897-5a03d148a482 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:49.196074 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-264e6bec-74d6-49ed-9897-5a03d148a482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:49.207851 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e6bec-74d6-49ed-9897-5a03d148a482 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:49.212286 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab87c21-5759-43bc-9d27-b37dc0001772 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:49.212941 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab87c21-5759-43bc-9d27-b37dc0001772 service cinder] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:49.213232 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab87c21-5759-43bc-9d27-b37dc0001772 service cinder] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:49.213515 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab87c21-5759-43bc-9d27-b37dc0001772 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:49.269944 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab87c21-5759-43bc-9d27-b37dc0001772 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:49.271842 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab87c21-5759-43bc-9d27-b37dc0001772 service cinder] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:49.281193 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1422/2865] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:16:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:16:49.827056 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46dc07f-1431-4d50-b311-302bf6e545d3 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:49.835874 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46dc07f-1431-4d50-b311-302bf6e545d3 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:49.839548 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840019f-cf61-4e34-b5f6-ae3fddea399d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:49.840191 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840019f-cf61-4e34-b5f6-ae3fddea399d service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:49.840479 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840019f-cf61-4e34-b5f6-ae3fddea399d service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:49.840761 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840019f-cf61-4e34-b5f6-ae3fddea399d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:49.850154 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840019f-cf61-4e34-b5f6-ae3fddea399d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:49.851615 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840019f-cf61-4e34-b5f6-ae3fddea399d service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:49.861151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1444/2866] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:16:49 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:16:53.739199 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab87c21-5759-43bc-9d27-b37dc0001772 service cinder] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:53.805746 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab87c21-5759-43bc-9d27-b37dc0001772 service cinder] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:53.810275 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e89bc-0dd6-47b5-958e-f7669f226fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:53.811144 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e89bc-0dd6-47b5-958e-f7669f226fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:53.811422 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e89bc-0dd6-47b5-958e-f7669f226fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:53.811690 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e89bc-0dd6-47b5-958e-f7669f226fdd admin admin] PATH_INFO: `/v3/users/517c37ed55804482b3914651af9b0af4` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:53.812858 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e89bc-0dd6-47b5-958e-f7669f226fdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=517c37ed55804482b3914651af9b0af4)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:53.814802 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e89bc-0dd6-47b5-958e-f7669f226fdd admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:53.844797 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:53.863408 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-110e89bc-0dd6-47b5-958e-f7669f226fdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '517c37ed55804482b3914651af9b0af4'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:53.871308 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-110e89bc-0dd6-47b5-958e-f7669f226fdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '517c37ed55804482b3914651af9b0af4'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:53.872010 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-110e89bc-0dd6-47b5-958e-f7669f226fdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '517c37ed55804482b3914651af9b0af4'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:53.872605 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-110e89bc-0dd6-47b5-958e-f7669f226fdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '517c37ed55804482b3914651af9b0af4'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:53.879612 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-110e89bc-0dd6-47b5-958e-f7669f226fdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '517c37ed55804482b3914651af9b0af4'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:53.897827 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1423/2867] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:16:53 2022] DELETE /identity/v3/users/517c37ed55804482b3914651af9b0af4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:53.902696 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0118bd0-2fa3-4e92-989f-379774b1ed00 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:53.903202 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0118bd0-2fa3-4e92-989f-379774b1ed00 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:53.903305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0118bd0-2fa3-4e92-989f-379774b1ed00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:53.920676 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e0118bd0-2fa3-4e92-989f-379774b1ed00 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:53.921864 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e0118bd0-2fa3-4e92-989f-379774b1ed00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:53.983305 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1445/2868] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:53.993640 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e89bc-0dd6-47b5-958e-f7669f226fdd admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:53.995098 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-110e89bc-0dd6-47b5-958e-f7669f226fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:54.051058 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e89bc-0dd6-47b5-958e-f7669f226fdd admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:54.055824 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46ae7d-5a3c-42d0-8c7e-81ca26289479 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:54.056475 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46ae7d-5a3c-42d0-8c7e-81ca26289479 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:54.056760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46ae7d-5a3c-42d0-8c7e-81ca26289479 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:54.057044 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46ae7d-5a3c-42d0-8c7e-81ca26289479 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:54.066346 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb46ae7d-5a3c-42d0-8c7e-81ca26289479 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:54.067780 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb46ae7d-5a3c-42d0-8c7e-81ca26289479 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:54.077596 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1424/2869] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:16:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:16:54.356898 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0118bd0-2fa3-4e92-989f-379774b1ed00 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:54.358714 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e0118bd0-2fa3-4e92-989f-379774b1ed00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:54.372533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0118bd0-2fa3-4e92-989f-379774b1ed00 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:54.377247 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c112f-643d-4e8c-949b-f033775a3f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:54.378641 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c112f-643d-4e8c-949b-f033775a3f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:54.378927 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c112f-643d-4e8c-949b-f033775a3f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:54.379390 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c112f-643d-4e8c-949b-f033775a3f51 admin admin] PATH_INFO: `/v3/projects/4db395eeabf54bfba70236de291c2044` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:54.380782 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c112f-643d-4e8c-949b-f033775a3f51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4db395eeabf54bfba70236de291c2044)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:54.382616 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c112f-643d-4e8c-949b-f033775a3f51 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:54.431670 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-522c112f-643d-4e8c-949b-f033775a3f51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4db395eeabf54bfba70236de291c2044', 'request_id': 'req-522c112f-643d-4e8c-949b-f033775a3f51'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:54.437159 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-522c112f-643d-4e8c-949b-f033775a3f51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4db395eeabf54bfba70236de291c2044', 'request_id': 'req-522c112f-643d-4e8c-949b-f033775a3f51'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:54.443630 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-522c112f-643d-4e8c-949b-f033775a3f51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4db395eeabf54bfba70236de291c2044', 'request_id': 'req-522c112f-643d-4e8c-949b-f033775a3f51'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:54.457221 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-522c112f-643d-4e8c-949b-f033775a3f51 admin admin] The token cache is being invalidate because project 4db395eeabf54bfba70236de291c2044 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:16:54.457507 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-522c112f-643d-4e8c-949b-f033775a3f51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:54.458151 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-522c112f-643d-4e8c-949b-f033775a3f51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:54.460589 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1446/2870] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:16:54 2022] DELETE /identity/v3/projects/4db395eeabf54bfba70236de291c2044 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:54.464646 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46ae7d-5a3c-42d0-8c7e-81ca26289479 service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:54.475111 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46ae7d-5a3c-42d0-8c7e-81ca26289479 service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:54.530484 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992476b1-43a2-448c-a446-0ef48fe597c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:54.531359 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992476b1-43a2-448c-a446-0ef48fe597c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:54.531648 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992476b1-43a2-448c-a446-0ef48fe597c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:54.531931 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992476b1-43a2-448c-a446-0ef48fe597c7 admin admin] PATH_INFO: `/v3/users/9b44db3704ad49cf93c433b16cfbffc6` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:54.533121 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992476b1-43a2-448c-a446-0ef48fe597c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b44db3704ad49cf93c433b16cfbffc6)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:54.535081 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992476b1-43a2-448c-a446-0ef48fe597c7 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:54.586236 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-992476b1-43a2-448c-a446-0ef48fe597c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b44db3704ad49cf93c433b16cfbffc6'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:54.587902 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:54.594814 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-992476b1-43a2-448c-a446-0ef48fe597c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b44db3704ad49cf93c433b16cfbffc6'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:54.595519 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-992476b1-43a2-448c-a446-0ef48fe597c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b44db3704ad49cf93c433b16cfbffc6'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:54.596153 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-992476b1-43a2-448c-a446-0ef48fe597c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b44db3704ad49cf93c433b16cfbffc6'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:54.604581 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-992476b1-43a2-448c-a446-0ef48fe597c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b44db3704ad49cf93c433b16cfbffc6'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:54.623354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1425/2871] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:16:54 2022] DELETE /identity/v3/users/9b44db3704ad49cf93c433b16cfbffc6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:54.897823 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c112f-643d-4e8c-949b-f033775a3f51 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:54.899570 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-522c112f-643d-4e8c-949b-f033775a3f51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:54.914128 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c112f-643d-4e8c-949b-f033775a3f51 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:54.964955 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:54.976478 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e977a-705d-40e6-9472-07213e94f337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:54.977300 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e977a-705d-40e6-9472-07213e94f337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:54.977785 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e977a-705d-40e6-9472-07213e94f337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:54.978308 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e977a-705d-40e6-9472-07213e94f337 admin admin] PATH_INFO: `/v3/projects/81e71a0f586a46d0b5239023599e6de4` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:54.979533 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e977a-705d-40e6-9472-07213e94f337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81e71a0f586a46d0b5239023599e6de4)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:54.981535 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e977a-705d-40e6-9472-07213e94f337 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:55.029179 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-146e977a-705d-40e6-9472-07213e94f337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81e71a0f586a46d0b5239023599e6de4', 'request_id': 'req-146e977a-705d-40e6-9472-07213e94f337'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:55.034943 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-146e977a-705d-40e6-9472-07213e94f337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81e71a0f586a46d0b5239023599e6de4', 'request_id': 'req-146e977a-705d-40e6-9472-07213e94f337'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:55.041505 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-146e977a-705d-40e6-9472-07213e94f337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81e71a0f586a46d0b5239023599e6de4', 'request_id': 'req-146e977a-705d-40e6-9472-07213e94f337'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:55.054787 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-146e977a-705d-40e6-9472-07213e94f337 admin admin] The token cache is being invalidate because project 81e71a0f586a46d0b5239023599e6de4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:16:55.055082 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-146e977a-705d-40e6-9472-07213e94f337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:55.055718 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-146e977a-705d-40e6-9472-07213e94f337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:55.057307 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1447/2872] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:16:54 2022] DELETE /identity/v3/projects/81e71a0f586a46d0b5239023599e6de4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:55.061510 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992476b1-43a2-448c-a446-0ef48fe597c7 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:55.062966 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-992476b1-43a2-448c-a446-0ef48fe597c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:55.077453 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992476b1-43a2-448c-a446-0ef48fe597c7 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:55.134465 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065a8f9-acd8-4882-a88e-60d74129b7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:55.135356 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065a8f9-acd8-4882-a88e-60d74129b7d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:55.135728 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065a8f9-acd8-4882-a88e-60d74129b7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:55.135948 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065a8f9-acd8-4882-a88e-60d74129b7d0 admin admin] PATH_INFO: `/v3/users/063465380cb94adcacb15d68ced98ff9` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:55.137170 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065a8f9-acd8-4882-a88e-60d74129b7d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=063465380cb94adcacb15d68ced98ff9)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:55.139147 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065a8f9-acd8-4882-a88e-60d74129b7d0 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:55.188678 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6065a8f9-acd8-4882-a88e-60d74129b7d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '063465380cb94adcacb15d68ced98ff9'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:55.196400 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6065a8f9-acd8-4882-a88e-60d74129b7d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '063465380cb94adcacb15d68ced98ff9'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:55.197151 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6065a8f9-acd8-4882-a88e-60d74129b7d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '063465380cb94adcacb15d68ced98ff9'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:55.197752 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6065a8f9-acd8-4882-a88e-60d74129b7d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '063465380cb94adcacb15d68ced98ff9'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:55.204724 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-6065a8f9-acd8-4882-a88e-60d74129b7d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '063465380cb94adcacb15d68ced98ff9'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:55.222081 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1426/2873] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:16:55 2022] DELETE /identity/v3/users/063465380cb94adcacb15d68ced98ff9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:16:55.501033 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e977a-705d-40e6-9472-07213e94f337 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:55.502567 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-146e977a-705d-40e6-9472-07213e94f337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:55.515883 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e977a-705d-40e6-9472-07213e94f337 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:55.575378 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8601a7-a0b3-4204-a14e-6b0c0f90ed23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:55.576251 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8601a7-a0b3-4204-a14e-6b0c0f90ed23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:55.576538 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8601a7-a0b3-4204-a14e-6b0c0f90ed23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:55.576817 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8601a7-a0b3-4204-a14e-6b0c0f90ed23 admin admin] PATH_INFO: `/v3/projects/8ffe788b6eeb49d6a9eee4d7378e119f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:55.577994 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8601a7-a0b3-4204-a14e-6b0c0f90ed23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ffe788b6eeb49d6a9eee4d7378e119f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:55.579957 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8601a7-a0b3-4204-a14e-6b0c0f90ed23 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:55.627395 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9c8601a7-a0b3-4204-a14e-6b0c0f90ed23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ffe788b6eeb49d6a9eee4d7378e119f', 'request_id': 'req-9c8601a7-a0b3-4204-a14e-6b0c0f90ed23'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:55.633667 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9c8601a7-a0b3-4204-a14e-6b0c0f90ed23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ffe788b6eeb49d6a9eee4d7378e119f', 'request_id': 'req-9c8601a7-a0b3-4204-a14e-6b0c0f90ed23'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:55.640212 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9c8601a7-a0b3-4204-a14e-6b0c0f90ed23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ffe788b6eeb49d6a9eee4d7378e119f', 'request_id': 'req-9c8601a7-a0b3-4204-a14e-6b0c0f90ed23'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:55.653054 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9c8601a7-a0b3-4204-a14e-6b0c0f90ed23 admin admin] The token cache is being invalidate because project 8ffe788b6eeb49d6a9eee4d7378e119f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:16:55.653363 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9c8601a7-a0b3-4204-a14e-6b0c0f90ed23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:55.654018 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9c8601a7-a0b3-4204-a14e-6b0c0f90ed23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:16:55.655610 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1448/2874] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:16:55 2022] DELETE /identity/v3/projects/8ffe788b6eeb49d6a9eee4d7378e119f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:55.661319 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a39f6-5a77-4909-9c99-23b66f73c759 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:55.661612 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a39f6-5a77-4909-9c99-23b66f73c759 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:55.662613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a39f6-5a77-4909-9c99-23b66f73c759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:55.682167 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-ec5a39f6-5a77-4909-9c99-23b66f73c759 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:55.683416 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-ec5a39f6-5a77-4909-9c99-23b66f73c759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:55.723313 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:55.745815 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1427/2875] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:55.807170 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f376a2-f875-4f88-a7d9-c40a7d6f8a9c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:55.807433 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f376a2-f875-4f88-a7d9-c40a7d6f8a9c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:55.807707 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f376a2-f875-4f88-a7d9-c40a7d6f8a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:55.827365 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-13f376a2-f875-4f88-a7d9-c40a7d6f8a9c None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:55.828564 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-13f376a2-f875-4f88-a7d9-c40a7d6f8a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:55.837654 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1449/2876] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:55.841886 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a39f6-5a77-4909-9c99-23b66f73c759 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:55.852533 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a39f6-5a77-4909-9c99-23b66f73c759 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:55.857169 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85058c0-15c9-4ef2-927e-b472094fca6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:55.857832 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85058c0-15c9-4ef2-927e-b472094fca6f admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:55.858123 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85058c0-15c9-4ef2-927e-b472094fca6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:55.858434 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85058c0-15c9-4ef2-927e-b472094fca6f admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:55.859193 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85058c0-15c9-4ef2-927e-b472094fca6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:16:55.859532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85058c0-15c9-4ef2-927e-b472094fca6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:55.861133 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85058c0-15c9-4ef2-927e-b472094fca6f admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:55.872768 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1428/2877] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:16:55.876940 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f376a2-f875-4f88-a7d9-c40a7d6f8a9c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:55.886480 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f376a2-f875-4f88-a7d9-c40a7d6f8a9c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:55.891203 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b40c8a-96b1-4dd3-868b-e1562f8edf30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:55.891861 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b40c8a-96b1-4dd3-868b-e1562f8edf30 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:55.892145 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b40c8a-96b1-4dd3-868b-e1562f8edf30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:55.892421 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b40c8a-96b1-4dd3-868b-e1562f8edf30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:55.893121 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b40c8a-96b1-4dd3-868b-e1562f8edf30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:55.895063 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b40c8a-96b1-4dd3-868b-e1562f8edf30 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:55.899280 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:16:55.924078 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1450/2878] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:16:55 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:16:55.928471 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85058c0-15c9-4ef2-927e-b472094fca6f admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:55.938443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85058c0-15c9-4ef2-927e-b472094fca6f admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:55.996562 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1e602-e955-4962-8526-0d80cb7b3e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:55.997152 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1e602-e955-4962-8526-0d80cb7b3e92 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:55.997443 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1e602-e955-4962-8526-0d80cb7b3e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:55.997720 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1e602-e955-4962-8526-0d80cb7b3e92 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:55.998458 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1e602-e955-4962-8526-0d80cb7b3e92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:56.000373 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1e602-e955-4962-8526-0d80cb7b3e92 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:56.019723 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1429/2879] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:16:55 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 18:16:56.024789 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b40c8a-96b1-4dd3-868b-e1562f8edf30 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:56.035074 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b40c8a-96b1-4dd3-868b-e1562f8edf30 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:56.040299 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065d06e-7857-489c-9c46-91123339b713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:56.040962 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065d06e-7857-489c-9c46-91123339b713 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:56.041248 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065d06e-7857-489c-9c46-91123339b713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:56.041523 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065d06e-7857-489c-9c46-91123339b713 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:56.042286 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e065d06e-7857-489c-9c46-91123339b713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:56.043888 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e065d06e-7857-489c-9c46-91123339b713 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:56.054799 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1451/2880] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:16:56.058718 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1e602-e955-4962-8526-0d80cb7b3e92 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:56.069030 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1e602-e955-4962-8526-0d80cb7b3e92 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:56.074120 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ddec6-bcb1-4aa8-b2d8-38f90a89c04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:56.074969 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ddec6-bcb1-4aa8-b2d8-38f90a89c04b admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:56.075253 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ddec6-bcb1-4aa8-b2d8-38f90a89c04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:56.075532 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ddec6-bcb1-4aa8-b2d8-38f90a89c04b admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:56.076259 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ddec6-bcb1-4aa8-b2d8-38f90a89c04b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:56.077852 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ddec6-bcb1-4aa8-b2d8-38f90a89c04b admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:56.088964 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1430/2881] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:16:56.093428 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065d06e-7857-489c-9c46-91123339b713 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:56.104042 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065d06e-7857-489c-9c46-91123339b713 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:56.108807 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7ad66-5dea-45b0-bbdf-d2246b78642b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:56.109683 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7ad66-5dea-45b0-bbdf-d2246b78642b admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:56.110001 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7ad66-5dea-45b0-bbdf-d2246b78642b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:56.110278 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7ad66-5dea-45b0-bbdf-d2246b78642b admin admin] PATH_INFO: `/v3/projects/c65de19dbca548959d62d3c454a7db07/users/100dde279e78417d8b3778e661d0f3c8/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:56.124741 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c7ad66-5dea-45b0-bbdf-d2246b78642b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c65de19dbca548959d62d3c454a7db07, user_id=100dde279e78417d8b3778e661d0f3c8, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:56.127485 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c7ad66-5dea-45b0-bbdf-d2246b78642b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:56.141242 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1452/2882] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:16:56 2022] PUT /identity/v3/projects/c65de19dbca548959d62d3c454a7db07/users/100dde279e78417d8b3778e661d0f3c8/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:16:56.152099 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26d6c2-8eac-4439-8dad-4a57d2b656cc None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:56.152391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26d6c2-8eac-4439-8dad-4a57d2b656cc None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:56.152703 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26d6c2-8eac-4439-8dad-4a57d2b656cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:56.180125 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-eb26d6c2-8eac-4439-8dad-4a57d2b656cc None None] MFA Rules not processed for user `100dde279e78417d8b3778e661d0f3c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:16:56.181333 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-eb26d6c2-8eac-4439-8dad-4a57d2b656cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:56.246410 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1431/2883] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:16:56 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:16:56.259454 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7ad66-5dea-45b0-bbdf-d2246b78642b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:16:56.261195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d5c7ad66-5dea-45b0-bbdf-d2246b78642b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:16:56.320664 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7ad66-5dea-45b0-bbdf-d2246b78642b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:16:56.324605 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d465d9e-1315-4e6e-ab3a-ab47242290c8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f4c9fc949c4a2c951b755234848cc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:16:56.325259 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d465d9e-1315-4e6e-ab3a-ab47242290c8 service glance] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:16:56.325544 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d465d9e-1315-4e6e-ab3a-ab47242290c8 service glance] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:16:56.325821 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d465d9e-1315-4e6e-ab3a-ab47242290c8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:16:56.334904 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d465d9e-1315-4e6e-ab3a-ab47242290c8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:16:56.336337 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d465d9e-1315-4e6e-ab3a-ab47242290c8 service glance] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:16:56.345276 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1453/2884] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:16:56 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:06.243421 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:06.341221 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:26.428493 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26d6c2-8eac-4439-8dad-4a57d2b656cc None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:26.491626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26d6c2-8eac-4439-8dad-4a57d2b656cc None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:26.496354 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abbeba-c0bb-42d2-bc09-91a4ec54510b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:26.497230 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abbeba-c0bb-42d2-bc09-91a4ec54510b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:26.497514 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abbeba-c0bb-42d2-bc09-91a4ec54510b admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:26.497791 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abbeba-c0bb-42d2-bc09-91a4ec54510b admin admin] PATH_INFO: `/v3/users/100dde279e78417d8b3778e661d0f3c8` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:26.498999 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71abbeba-c0bb-42d2-bc09-91a4ec54510b admin admin] RBAC: Authorizing `identity:delete_user(user_id=100dde279e78417d8b3778e661d0f3c8)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:26.500931 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71abbeba-c0bb-42d2-bc09-91a4ec54510b admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:26.547433 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-71abbeba-c0bb-42d2-bc09-91a4ec54510b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '100dde279e78417d8b3778e661d0f3c8'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:26.555733 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-71abbeba-c0bb-42d2-bc09-91a4ec54510b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '100dde279e78417d8b3778e661d0f3c8'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:26.556390 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-71abbeba-c0bb-42d2-bc09-91a4ec54510b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '100dde279e78417d8b3778e661d0f3c8'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:26.557008 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-71abbeba-c0bb-42d2-bc09-91a4ec54510b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '100dde279e78417d8b3778e661d0f3c8'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:26.564057 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-71abbeba-c0bb-42d2-bc09-91a4ec54510b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '100dde279e78417d8b3778e661d0f3c8'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:26.581277 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1432/2885] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:17:26 2022] DELETE /identity/v3/users/100dde279e78417d8b3778e661d0f3c8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:26.586772 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4b62c-21d7-40e7-9341-b3d0b163e820 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:26.587336 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4b62c-21d7-40e7-9341-b3d0b163e820 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:26.587336 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4b62c-21d7-40e7-9341-b3d0b163e820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:26.606086 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a1b4b62c-21d7-40e7-9341-b3d0b163e820 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:26.607363 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a1b4b62c-21d7-40e7-9341-b3d0b163e820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:26.671254 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1454/2886] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:17:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:26.681011 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abbeba-c0bb-42d2-bc09-91a4ec54510b admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:26.682459 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-71abbeba-c0bb-42d2-bc09-91a4ec54510b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:26.741252 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abbeba-c0bb-42d2-bc09-91a4ec54510b admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:26.744930 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d0d6b-041d-48fb-b0dd-87abaa066e9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:26.745568 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d0d6b-041d-48fb-b0dd-87abaa066e9b service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:26.745853 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d0d6b-041d-48fb-b0dd-87abaa066e9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:26.746133 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d0d6b-041d-48fb-b0dd-87abaa066e9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:26.755376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d0d6b-041d-48fb-b0dd-87abaa066e9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:26.756810 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d0d6b-041d-48fb-b0dd-87abaa066e9b service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:26.766305 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1433/2887] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:17:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:27.063329 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4b62c-21d7-40e7-9341-b3d0b163e820 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:27.064811 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a1b4b62c-21d7-40e7-9341-b3d0b163e820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:27.083581 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4b62c-21d7-40e7-9341-b3d0b163e820 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:27.088693 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594e800-af30-4bcc-9080-f83085cee620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:27.091045 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594e800-af30-4bcc-9080-f83085cee620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:27.091045 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594e800-af30-4bcc-9080-f83085cee620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:27.091045 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594e800-af30-4bcc-9080-f83085cee620 admin admin] PATH_INFO: `/v3/projects/c65de19dbca548959d62d3c454a7db07` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:27.095272 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594e800-af30-4bcc-9080-f83085cee620 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c65de19dbca548959d62d3c454a7db07)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:27.095272 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594e800-af30-4bcc-9080-f83085cee620 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:27.152109 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8594e800-af30-4bcc-9080-f83085cee620 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c65de19dbca548959d62d3c454a7db07', 'request_id': 'req-8594e800-af30-4bcc-9080-f83085cee620'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:27.158855 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8594e800-af30-4bcc-9080-f83085cee620 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c65de19dbca548959d62d3c454a7db07', 'request_id': 'req-8594e800-af30-4bcc-9080-f83085cee620'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:27.166657 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8594e800-af30-4bcc-9080-f83085cee620 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c65de19dbca548959d62d3c454a7db07', 'request_id': 'req-8594e800-af30-4bcc-9080-f83085cee620'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:27.180470 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8594e800-af30-4bcc-9080-f83085cee620 admin admin] The token cache is being invalidate because project c65de19dbca548959d62d3c454a7db07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:17:27.180690 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8594e800-af30-4bcc-9080-f83085cee620 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:27.181320 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8594e800-af30-4bcc-9080-f83085cee620 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:27.182919 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1455/2888] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:17:27 2022] DELETE /identity/v3/projects/c65de19dbca548959d62d3c454a7db07 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:27.187907 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5ae9c-d11f-4bfd-bf23-8b9d44f69c07 None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:27.188278 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5ae9c-d11f-4bfd-bf23-8b9d44f69c07 None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:27.188628 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5ae9c-d11f-4bfd-bf23-8b9d44f69c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:27.209004 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-0ca5ae9c-d11f-4bfd-bf23-8b9d44f69c07 None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:27.210318 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-0ca5ae9c-d11f-4bfd-bf23-8b9d44f69c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:27.228332 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:27.281963 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1434/2889] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:17:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 18:17:27.294906 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74e751-9577-400e-aebe-ef3e98e8a7bf None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:27.295195 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74e751-9577-400e-aebe-ef3e98e8a7bf None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:27.295474 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74e751-9577-400e-aebe-ef3e98e8a7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:27.318075 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6e74e751-9577-400e-aebe-ef3e98e8a7bf None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:27.319399 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6e74e751-9577-400e-aebe-ef3e98e8a7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:27.328541 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1456/2890] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:17:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:27.332962 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5ae9c-d11f-4bfd-bf23-8b9d44f69c07 None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:27.342986 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5ae9c-d11f-4bfd-bf23-8b9d44f69c07 None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:27.347760 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2f440-b72d-4d24-9ff7-441fe01bbed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:27.348779 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2f440-b72d-4d24-9ff7-441fe01bbed2 admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:27.349076 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2f440-b72d-4d24-9ff7-441fe01bbed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:27.349351 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2f440-b72d-4d24-9ff7-441fe01bbed2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:27.350086 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2f440-b72d-4d24-9ff7-441fe01bbed2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80260) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 18:17:27.350420 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2f440-b72d-4d24-9ff7-441fe01bbed2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:27.352089 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2f440-b72d-4d24-9ff7-441fe01bbed2 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:27.365360 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1435/2891] 10.222.0.76 () {58 vars in 1152 bytes} [Thu May 26 18:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 18:17:27.370108 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74e751-9577-400e-aebe-ef3e98e8a7bf None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:27.380613 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74e751-9577-400e-aebe-ef3e98e8a7bf None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:27.386925 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf5d12-7f3c-4d25-b632-3f91819d95c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:27.388757 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf5d12-7f3c-4d25-b632-3f91819d95c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:27.389043 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf5d12-7f3c-4d25-b632-3f91819d95c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:27.389323 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf5d12-7f3c-4d25-b632-3f91819d95c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:27.390059 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf5d12-7f3c-4d25-b632-3f91819d95c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:27.392014 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf5d12-7f3c-4d25-b632-3f91819d95c5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:27.425778 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1457/2892] 10.222.0.76 () {60 vars in 1152 bytes} [Thu May 26 18:17:27 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:27.431626 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2f440-b72d-4d24-9ff7-441fe01bbed2 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:27.441718 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2f440-b72d-4d24-9ff7-441fe01bbed2 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:27.506416 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932bf9d-6252-422a-a403-a62208ea3a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:27.507043 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932bf9d-6252-422a-a403-a62208ea3a61 admin admin] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:27.507341 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932bf9d-6252-422a-a403-a62208ea3a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:27.507613 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932bf9d-6252-422a-a403-a62208ea3a61 admin admin] PATH_INFO: `/v3/users` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:27.509157 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932bf9d-6252-422a-a403-a62208ea3a61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:27.511543 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932bf9d-6252-422a-a403-a62208ea3a61 admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:27.537463 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1436/2893] 10.222.0.76 () {60 vars in 1143 bytes} [Thu May 26 18:17:27 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 18:17:27.541658 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf5d12-7f3c-4d25-b632-3f91819d95c5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:27.555347 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf5d12-7f3c-4d25-b632-3f91819d95c5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:27.563675 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4c9b4-249b-4de0-b98f-1a438cf30ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:27.564380 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4c9b4-249b-4de0-b98f-1a438cf30ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:27.564666 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4c9b4-249b-4de0-b98f-1a438cf30ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:27.564943 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4c9b4-249b-4de0-b98f-1a438cf30ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:27.565678 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4c9b4-249b-4de0-b98f-1a438cf30ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:27.569386 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4c9b4-249b-4de0-b98f-1a438cf30ca6 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:27.581138 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1458/2894] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:27.585914 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932bf9d-6252-422a-a403-a62208ea3a61 admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:27.596739 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932bf9d-6252-422a-a403-a62208ea3a61 admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:27.603725 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ce887-1743-4c62-8825-b89508f4733e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:27.604408 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ce887-1743-4c62-8825-b89508f4733e admin admin] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:27.605577 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ce887-1743-4c62-8825-b89508f4733e admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:27.605865 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ce887-1743-4c62-8825-b89508f4733e admin admin] PATH_INFO: `/v3/roles` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:27.606684 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ce887-1743-4c62-8825-b89508f4733e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:27.608452 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ce887-1743-4c62-8825-b89508f4733e admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:27.621792 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1437/2895] 10.222.0.76 () {58 vars in 1121 bytes} [Thu May 26 18:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 18:17:27.626005 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4c9b4-249b-4de0-b98f-1a438cf30ca6 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:27.637107 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4c9b4-249b-4de0-b98f-1a438cf30ca6 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:27.642073 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0dc3db-be8d-4b14-8edc-fddab3401628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:27.643057 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dc3db-be8d-4b14-8edc-fddab3401628 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:27.643343 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dc3db-be8d-4b14-8edc-fddab3401628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:27.643623 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dc3db-be8d-4b14-8edc-fddab3401628 admin admin] PATH_INFO: `/v3/projects/4c0cc426e4b148d89e274e597ceb72fa/users/1d7d1cc116bd4e33956ff22a04b69817/roles/eb1858a5621543028836223abcb3c0a1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:27.659318 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0dc3db-be8d-4b14-8edc-fddab3401628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c0cc426e4b148d89e274e597ceb72fa, user_id=1d7d1cc116bd4e33956ff22a04b69817, role_id=eb1858a5621543028836223abcb3c0a1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:27.662179 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0dc3db-be8d-4b14-8edc-fddab3401628 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:27.678269 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1459/2896] 10.222.0.76 () {60 vars in 1482 bytes} [Thu May 26 18:17:27 2022] PUT /identity/v3/projects/4c0cc426e4b148d89e274e597ceb72fa/users/1d7d1cc116bd4e33956ff22a04b69817/roles/eb1858a5621543028836223abcb3c0a1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:27.689336 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6c485-cdf6-41fc-87d1-1852ba0bb95e None None] REQUEST_METHOD: `POST` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:27.689555 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6c485-cdf6-41fc-87d1-1852ba0bb95e None None] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:27.689836 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6c485-cdf6-41fc-87d1-1852ba0bb95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:27.718376 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.auth.core [None req-51f6c485-cdf6-41fc-87d1-1852ba0bb95e None None] MFA Rules not processed for user `1d7d1cc116bd4e33956ff22a04b69817`. Rule list: `[]` (Enabled: `True`). {{(pid=80260) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:27.719604 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-51f6c485-cdf6-41fc-87d1-1852ba0bb95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:27.791946 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1438/2897] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:17:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:27.809475 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0dc3db-be8d-4b14-8edc-fddab3401628 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:27.811196 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0c0dc3db-be8d-4b14-8edc-fddab3401628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:27.834775 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:27.899584 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0dc3db-be8d-4b14-8edc-fddab3401628 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:27.903375 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d658c-28dc-4745-a3c1-a9839afeb7d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76dbf8f3f1b44baca2d2aa90fe606a30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:27.904026 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d658c-28dc-4745-a3c1-a9839afeb7d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:27.904318 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d658c-28dc-4745-a3c1-a9839afeb7d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:27.904599 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d658c-28dc-4745-a3c1-a9839afeb7d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:27.913823 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d658c-28dc-4745-a3c1-a9839afeb7d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:27.915298 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d658c-28dc-4745-a3c1-a9839afeb7d8 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:27.924274 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1460/2898] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:17:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:30.471523 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6c485-cdf6-41fc-87d1-1852ba0bb95e None None] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:30.491188 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:30.537770 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6c485-cdf6-41fc-87d1-1852ba0bb95e None None] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:30.542435 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194f242-1974-429d-b36d-6a9254e74fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:30.543316 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194f242-1974-429d-b36d-6a9254e74fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:30.543608 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194f242-1974-429d-b36d-6a9254e74fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:30.543887 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194f242-1974-429d-b36d-6a9254e74fac admin admin] PATH_INFO: `/v3/users/1d7d1cc116bd4e33956ff22a04b69817` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:30.545112 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194f242-1974-429d-b36d-6a9254e74fac admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d7d1cc116bd4e33956ff22a04b69817)` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:30.547088 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194f242-1974-429d-b36d-6a9254e74fac admin admin] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:30.594743 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d194f242-1974-429d-b36d-6a9254e74fac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d7d1cc116bd4e33956ff22a04b69817'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:30.602959 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d194f242-1974-429d-b36d-6a9254e74fac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d7d1cc116bd4e33956ff22a04b69817'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:30.603568 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d194f242-1974-429d-b36d-6a9254e74fac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d7d1cc116bd4e33956ff22a04b69817'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:30.604161 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d194f242-1974-429d-b36d-6a9254e74fac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d7d1cc116bd4e33956ff22a04b69817'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:30.611123 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.notifications [None req-d194f242-1974-429d-b36d-6a9254e74fac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d7d1cc116bd4e33956ff22a04b69817'} {{(pid=80260) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:30.628141 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1439/2899] 10.222.0.76 () {58 vars in 1223 bytes} [Thu May 26 18:17:30 2022] DELETE /identity/v3/users/1d7d1cc116bd4e33956ff22a04b69817 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 18:17:30.633194 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35170d6c-b90d-436f-b162-75e71be0e39e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:30.633690 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35170d6c-b90d-436f-b162-75e71be0e39e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:30.633793 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35170d6c-b90d-436f-b162-75e71be0e39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:30.653124 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-35170d6c-b90d-436f-b162-75e71be0e39e None None] MFA Rules not processed for user `3c37098f90fe47ab88491e90626539d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 18:17:30.654363 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-35170d6c-b90d-436f-b162-75e71be0e39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:30.714997 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1461/2900] 10.222.0.76 () {58 vars in 957 bytes} [Thu May 26 18:17:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 18:17:30.724682 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194f242-1974-429d-b36d-6a9254e74fac admin admin] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:30.726099 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-d194f242-1974-429d-b36d-6a9254e74fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:30.786860 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194f242-1974-429d-b36d-6a9254e74fac admin admin] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:30.790500 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab3195-a569-4a2e-9678-173ae6205fe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:30.791151 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab3195-a569-4a2e-9678-173ae6205fe5 service neutron] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:30.791434 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab3195-a569-4a2e-9678-173ae6205fe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:30.791713 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab3195-a569-4a2e-9678-173ae6205fe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:30.800726 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab3195-a569-4a2e-9678-173ae6205fe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:30.802727 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab3195-a569-4a2e-9678-173ae6205fe5 service neutron] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:30.812394 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1440/2901] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:17:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:17:31.083587 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35170d6c-b90d-436f-b162-75e71be0e39e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:17:31.085053 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-35170d6c-b90d-436f-b162-75e71be0e39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:17:31.097169 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35170d6c-b90d-436f-b162-75e71be0e39e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:17:31.102025 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114329ce-695b-47ae-b184-fb87f302d311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c37098f90fe47ab88491e90626539d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a106ee939504b208f8d5ca1deb61f2d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:17:31.102947 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114329ce-695b-47ae-b184-fb87f302d311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:17:31.103233 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114329ce-695b-47ae-b184-fb87f302d311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:17:31.103513 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114329ce-695b-47ae-b184-fb87f302d311 admin admin] PATH_INFO: `/v3/projects/4c0cc426e4b148d89e274e597ceb72fa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:17:31.104743 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114329ce-695b-47ae-b184-fb87f302d311 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c0cc426e4b148d89e274e597ceb72fa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:17:31.106696 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114329ce-695b-47ae-b184-fb87f302d311 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:17:31.163194 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-114329ce-695b-47ae-b184-fb87f302d311 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c0cc426e4b148d89e274e597ceb72fa', 'request_id': 'req-114329ce-695b-47ae-b184-fb87f302d311'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:31.169164 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-114329ce-695b-47ae-b184-fb87f302d311 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c0cc426e4b148d89e274e597ceb72fa', 'request_id': 'req-114329ce-695b-47ae-b184-fb87f302d311'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:31.175528 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-114329ce-695b-47ae-b184-fb87f302d311 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c0cc426e4b148d89e274e597ceb72fa', 'request_id': 'req-114329ce-695b-47ae-b184-fb87f302d311'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:31.189230 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-114329ce-695b-47ae-b184-fb87f302d311 admin admin] The token cache is being invalidate because project 4c0cc426e4b148d89e274e597ceb72fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 18:17:31.189515 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-114329ce-695b-47ae-b184-fb87f302d311 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:31.190134 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-114329ce-695b-47ae-b184-fb87f302d311 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 18:17:31.191849 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1462/2902] 10.222.0.76 () {58 vars in 1232 bytes} [Thu May 26 18:17:31 2022] DELETE /identity/v3/projects/4c0cc426e4b148d89e274e597ceb72fa => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 18:17:40.808076 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:17:41.186168 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:18:52.132906 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab3195-a569-4a2e-9678-173ae6205fe5 service neutron] Authenticating user token {{(pid=80260) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:18:52.134391 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-feab3195-a569-4a2e-9678-173ae6205fe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:52.229492 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab3195-a569-4a2e-9678-173ae6205fe5 service neutron] Validating token access rules against request {{(pid=80260) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:18:52.291588 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4ec55-845d-4554-8aeb-8f72df8e16cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '596f126382e3440fb3cf7c357eef889f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80260) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:18:52.292169 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4ec55-845d-4554-8aeb-8f72df8e16cd service placement] REQUEST_METHOD: `GET` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:18:52.292459 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4ec55-845d-4554-8aeb-8f72df8e16cd service placement] SCRIPT_NAME: `/identity` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:18:52.292738 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4ec55-845d-4554-8aeb-8f72df8e16cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80260) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:18:52.294608 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.fernet_utils [None req-a1d4ec55-845d-4554-8aeb-8f72df8e16cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80260) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:18:52.306929 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4ec55-845d-4554-8aeb-8f72df8e16cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:18:52.308554 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4ec55-845d-4554-8aeb-8f72df8e16cd service placement] RBAC: Authorization granted {{(pid=80260) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:18:52.319335 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: [pid: 80260|app: 0|req: 1441/2903] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:18:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:19:02.314178 dv-cinder-839064-2-iscsi devstack@keystone.service[80260]: DEBUG dbcounter [-] [80260] Writing DB stats keystone:SELECT=67 {{(pid=80260) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 18:20:55.494699 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114329ce-695b-47ae-b184-fb87f302d311 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 18:20:55.496157 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-114329ce-695b-47ae-b184-fb87f302d311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:55.553152 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114329ce-695b-47ae-b184-fb87f302d311 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 18:20:55.556724 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c53a0d-49c3-4d2f-9676-38d14905e55e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df66990a3734209a350c8b6768d259f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1d09f491d944f99425a641561b03c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 18:20:55.557373 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c53a0d-49c3-4d2f-9676-38d14905e55e service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 18:20:55.557660 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c53a0d-49c3-4d2f-9676-38d14905e55e service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 18:20:55.557939 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c53a0d-49c3-4d2f-9676-38d14905e55e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 18:20:55.559727 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b1c53a0d-49c3-4d2f-9676-38d14905e55e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 18:20:55.569749 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c53a0d-49c3-4d2f-9676-38d14905e55e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 18:20:55.571200 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c53a0d-49c3-4d2f-9676-38d14905e55e service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 18:20:55.579530 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1463/2904] 10.222.0.76 () {60 vars in 1365 bytes} [Thu May 26 18:20:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 18:21:05.575732 dv-cinder-839064-2-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=26 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}